This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
guides:slurm [23.09.2019 10:59] Teemu Kuulasmaa |
guides:slurm [15.11.2021 16:19] (current) Administrator |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | # SLURM Workload Manager | + | # Slurm Workload Manager |
- | [SLURM Workload Manager](https:// | + | [Slurm Workload Manager](https:// |
- | ### SLURM Partitions on sampo.uef.fi | + | {{: |
- | - **serial**. 4 out of 4 nodes. Maximum run time 3 days | + | ## Slurm Partitions on sampo.uef.fi |
- | - **longrun**. 2 out of 4 nodes. Maximum run time 14 days | + | |
- | - **parallel**. 2 of 4 nodes. Maximum run time 3 days. | + | |
- | ### Explanation | + | - **serial**. 4 out of 6 nodes. Maximum run time 3 days |
+ | - **longrun**. 2 out of 6 nodes. Maximum run time 14 days | ||
+ | - **parallel**. 2 of 6 nodes. Maximum run time 3 days. | ||
+ | - **gpu**. 2 nodes. Maximum run time 3 days. | ||
- | Compute nodes are grouped in multiple | + | ## Explanation of the partitions |
- | **Serial** partition is the default partition for all jobs that user submits. | + | Compute nodes are grouped into multiple partitions and each partition can be considered as a job queue. Partitions can have multiple constraints and restrictions. For example access to certain partitions can be limited by the user/group or the maximum running time can restricted. |
+ | |||
+ | **Serial** partition is the default partition for all jobs that the user submits. | ||
**Longrun** partition is for long running jobs and only one node is for this usage. Default run time 5 minutes and maximum 14 days. | **Longrun** partition is for long running jobs and only one node is for this usage. Default run time 5 minutes and maximum 14 days. | ||
- | **Parallel** partition is for parallel jobs that can span over multiple nodes (MPI jobs for example). | + | **Parallel** partition is for parallel jobs that can span over multiple nodes (MPI jobs for example). |
- | + | ||
- | ### Using R with SLURM | + | |
- | + | ||
- | Example script (**hello.R**): | + | |
- | ``` | + | |
- | sayHello <- function(){ | + | |
- | print(" | + | |
- | } | + | |
- | sayHello() | + | |
- | ``` | + | |
- | + | ||
- | User can execute R scripts from the command line with the following commands: | + | |
- | + | ||
- | 1. R CMD BATCH script.R | + | |
- | 2. Rscript script.R | + | |
- | + | ||
- | Note: With the **R CMD BATCH** command the output of the R script is redirected to file instead of the screen | + | |
- | + | ||
- | Next user must embed the script to the **SLURM batch job file/ | + | |
- | + | ||
- | ``` | + | |
- | + | ||
- | # | + | |
- | #SBATCH --job-name helloworld # Name for your job | + | |
- | #SBATCH --ntasks 1 # Number of task | + | |
- | #SBATCH --time 5 # Runtime in minutes. | + | |
- | #SBATCH --mem=2000 # Reserve 2 GB RAM for the job | + | |
- | #SBATCH --partition serial # Partition to submit | + | |
- | #SBATCH --output hello.out # Standard out goes to this file | + | |
- | #SBATCH --error hello.err # Standard err goes to this file | + | |
- | #SBATCH --mail-user username@uef.fi # this is the email you wish to be notified at | + | |
- | #SBATCH --mail-type ALL # ALL will alert you of job beginning, completion, failure etc | + | |
- | + | ||
- | module load r # load modules | + | |
- | + | ||
- | Rscript hello.R # Execute the script | + | |
- | + | ||
- | ``` | + | |
- | + | ||
- | User can submit the job to the compute queue with the **[sbatch](https:// | + | |
- | + | ||
- | ``` | + | |
- | sbatch submit.sbatch | + | |
- | ``` | + | |
- | + | ||
- | User can monitor the progress of the job with the **[squeue](https:// | + | |
- | + | ||
- | ``` | + | |
- | squeue -j JOBID | + | |
- | ``` | + | |
- | + | ||
- | Also while the job is running user can login to executing compute node with the ssh command. When job is over the ssh session is terminated. | + | |
- | + | ||
- | ``` | + | |
- | ssh sampo1 | + | |
- | ``` | + | |
- | + | ||
- | ** Interactive session ** | + | |
- | + | ||
- | User can get an interactive sessions for whatever purpose. For this to be effective free node is more or less required. Following command will open bash session to any free node on the serial parallel for the next 5 minutes. | + | |
- | + | ||
- | ``` | + | |
- | srun -p serial --pty -t 0-00:05 /bin/bash | + | |
- | ``` | + | |
- | + | ||
- | ** Slurm job efficiency report (seff) and Accounting ** | + | |
- | + | ||
- | SLURM can provide the user with various job statistics. Like memory usage and CPU time. | + | |
- | for example with seff (Slurm job effiency report) it is possible to monitor on how efficiency the job was. | + | |
- | + | ||
- | ``` | + | |
- | seff JOBID | + | |
- | ``` | + | |
- | + | ||
- | It is particularly useful to add following line to the end of the sbatch script: | + | |
- | ``` | + | **GPU** partition is for gpu jobs (CUDA jobs). The user can reserve 2 nodes with 8xNVIDIA A100/40 GB. Default run time is 5 minutes and maximum 3 days. |
- | seff $SLURM_JOBID | + | |
- | ``` | + | |
- | or if you wish to have more detailed information | ||
- | ``` | ||
- | # show all own jobs contained in the accounting database | ||
- | sacct | ||
- | # show specific job | ||
- | sacct -j JOBID | ||
- | # specify fields | ||
- | sacct -j JOBID -o JobName, | ||
- | # show all fields | ||
- | sacct -j JOBID -o ALL | ||
- | ``` |