This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
guides:slurm:basics [23.09.2019 11:19] Teemu Kuulasmaa |
guides:slurm:basics [27.11.2023 12:40] (current) Administrator |
||
---|---|---|---|
Line 8: | Line 8: | ||
* Optional plugins can be used for accounting, advanced reservation, | * Optional plugins can be used for accounting, advanced reservation, | ||
- | Bioinformatics Center uses unmodified version of Slurm on sampo.uef.fi computing cluster. This guarantees that the most tutorials and guides found from the Internet are applicable. The obvious starting place to search for usage information is Slurm own website [Slurm Workload Manager](https:// | + | Bioinformatics Center uses unmodified version of Slurm on sampo.uef.fi computing cluster. This guarantees that the most of the tutorials and guides found from the Internet are applicable |
+ | |||
+ | {{ : | ||
+ | |||
+ | |||
+ | ### Example | ||
+ | |||
+ | In this example we will run simple [MATLAB](https:// | ||
+ | |||
+ | |||
+ | #### Example MATLAB code (matlab.m) | ||
+ | ``` | ||
+ | % Creates a 10x10 Magic square | ||
+ | M = magic(10); | ||
+ | M | ||
+ | ``` | ||
+ | #### Example script (submit.sbatch) | ||
+ | |||
+ | Here we have specified the [batch script](https:// | ||
- | ### Example script | ||
``` | ``` | ||
#!/bin/bash | #!/bin/bash | ||
- | #SBATCH --job-name helloworld # Name for your job | + | #SBATCH --ntasks 1 # Number of task |
- | #SBATCH --ntasks 1 # Number of task | + | #SBATCH --time 5 # Runtime in minutes. |
- | #SBATCH --time 5 # Runtime in minutes. | + | #SBATCH --mem=2000 # Reserve 2 GB RAM for the job |
- | #SBATCH --mem=2000 # Reserve 2 GB RAM for the job | + | #SBATCH --partition serial # Partition to submit |
- | #SBATCH --partition serial # Partition to submit | + | |
- | #SBATCH --output hello.out # Standard out goes to this file | + | |
- | #SBATCH --error hello.err # Standard err goes to this file | + | |
- | #SBATCH --mail-user username@uef.fi # this is the email you wish to be notified at | + | |
- | #SBATCH --mail-type ALL # ALL will alert you of job beginning, completion, failure etc | + | |
- | module load r # load modules | + | module load matlab/ |
- | Rscript hello.R # Execute the script | + | matlab -nodisplay < matlab.m # Execute the script |
``` | ``` | ||
- | User can submit the job to the compute queue with the **[sbatch](https:// | + | User can submit the job to the compute queue with the **[sbatch](https:// |
``` | ``` | ||
sbatch submit.sbatch | sbatch submit.sbatch | ||
``` | ``` | ||
+ | **[sbatch](https:// | ||
- | User can monitor the progress of the job with the **[squeue](https:// | + | Here is an another example on how to analyze variants. |
``` | ``` | ||
- | squeue | + | # |
+ | # | ||
+ | #SBATCH --time 00: | ||
+ | #SBATCH --mem 2000 # Reserve 2 GB RAM for the job | ||
+ | #SBATCH --partition serial | ||
+ | |||
+ | module load bcftools # load modules | ||
+ | |||
+ | # filter variants and calculate stats | ||
+ | bcftools filter --include' | ||
``` | ``` | ||
- | Also while the job is running user can login to executing compute node with the ssh command. When job is over the ssh session is terminated. | + | Submit |
``` | ``` | ||
- | ssh sampo1 | + | sbatch variants.sbatch |
``` | ``` | ||
- | ** Interactive session ** | ||
- | User can get an interactive sessions for whatever purpose. For this to be effective free node is more or less required. Following | + | ### Slurm job queue |
+ | User can monitor the state of the job with the **[squeue](https:// | ||
``` | ``` | ||
- | srun -p serial --pty -t 0-00:05 /bin/bash | + | squeue |
``` | ``` | ||
- | ** Slurm job efficiency report (seff) and Accounting ** | + | ### Output of the job is available in local output file. |
- | SLURM can provide the user with various job statistics. Like memory usage and CPU time. | + | ``` |
- | for example with seff (Slurm job effiency report) it is possible to monitor on how efficiency | + | less slurm-< |
+ | ``` | ||
+ | |||
+ | ### Scontrol - View or modify Slurm configuration | ||
+ | [Scontrol](https:// | ||
+ | #### List all compute nodes | ||
``` | ``` | ||
- | seff JOBID | + | scontrol show node |
``` | ``` | ||
- | It is particularly useful to add following line to the end of the sbatch script: | + | #### List all compute nodes |
+ | ``` | ||
+ | scontrol show node | ||
+ | ``` | ||
+ | #### List all queues/ | ||
``` | ``` | ||
- | seff $SLURM_JOBID | + | scontrol show partition |
``` | ``` | ||
- | or if you wish to have more detailed | + | #### List information |
+ | ``` | ||
+ | scontrol show job < | ||
+ | ``` | ||
+ | |||
+ | |||
+ | ### Slurm job effiency report (seff) | ||
+ | Seff command will give the report of the completed job on how much resources it consumed. The reported information are CPU wall time, job runtime and memory usage. | ||
``` | ``` | ||
- | # show all own jobs contained in the accounting database | + | seff < |
- | sacct | + | |
- | # show specific job | + | |
- | sacct -j JOBID | + | |
- | # specify fields | + | |
- | sacct -j JOBID -o JobName, | + | |
- | # show all fields | + | |
- | sacct -j JOBID -o ALL | + | |
``` | ``` | ||
+ | |||
+ |