Bioinformatics Center

User Tools

Site Tools

You are here: home » guides » slurm » basics
guides:slurm:basics

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision 		Previous revision
guides:slurm:basics [23.09.2019 11:04]
Teemu Kuulasmaa created 		guides:slurm:basics [27.11.2023 12:40] (current)
Administrator
Line 1: Line 1:
-# SLURM basics+# Slurm basics
  
-Example script (**hello.R**): +Slurm Workload manager is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. As a cluster workload manager, Slurm has three key functions. 
-``` +
-sayHello <- function(){ +
-  print("hello") +
-} +
-sayHello() +
-```+
  
-User can execute R scripts from the command line with the following commands:+  * It allocates exclusive and/or non-exclusive access to resources (compute nodes) to users for some duration of time so they can perform work. 
 +  * It provides a framework for starting, executing, and monitoring work (normally a parallel job) on the set of allocated nodes. 
 +  * It arbitrates contention for resources by managing a queue of pending work. 
 +  * Optional plugins can be used for accounting, advanced reservation, gang scheduling (time sharing for parallel jobs), backfill scheduling, topology optimized resource selection, resource limits by user or bank account, and sophisticated multifactor job prioritization algorithms.
  
-1. R CMD BATCH script.R +Bioinformatics Center uses unmodified version of Slurm on sampo.uef.fi computing cluster. This guarantees that the most of the tutorials and guides found from the Internet are applicable as-is. The most obvious starting place to search for usage information is documentation section of the Slurm own website [[https://slurm.schedmd.com|Slurm Workload Manager]].
-2. Rscript script.R+
  
-Note: With the **R CMD BATCH** command the output of the R script is redirected to file instead of the screen+{{ :guides:slurm:slurm.png |}}
  
-Next user must embed the script to the **SLURM batch job file/control file** (**submit.sbatch**):+ 
 +### Example 
 + 
 +In this example we will run simple [MATLAB](https://www.mathworks.com/products/matlab.html) code on one computing node. 
 + 
 + 
 +#### Example MATLAB code (matlab.m) 
 +``` 
 +% Creates a 10x10 Magic square 
 +M = magic(10); 
 +M 
 +``` 
 +#### Example script (submit.sbatch) 
 + 
 +Here we have specified the [batch script](https://slurm.schedmd.com/sbatch.html) with few basic [options](https://slurm.schedmd.com/sbatch.html#lbAG). It is important to reserve the amount of RAM that you'll need and estimate the runtime of your code. Optionally you can give name for your job.
  
 ``` ```
  
 #!/bin/bash #!/bin/bash
-#SBATCH --job-name helloworld # Name for your job +#SBATCH --ntasks 1 # Number of task 
-#SBATCH --ntasks 1 # Number of task +#SBATCH --time 5 # Runtime in minutes. 
-#SBATCH --time 5 # Runtime in minutes. +#SBATCH --mem=2000 # Reserve 2 GB RAM for the job 
-#SBATCH --mem=2000 # Reserve 2 GB RAM for the job +#SBATCH --partition serial # Partition to submit
-#SBATCH --partition serial # Partition to submit +
-#SBATCH --output hello.out # Standard out goes to this file +
-#SBATCH --error hello.err # Standard err goes to this file +
-#SBATCH --mail-user username@uef.fi # this is the email you wish to be notified at +
-#SBATCH --mail-type ALL # ALL will alert you of job beginning, completion, failure etc+
  
-module load r # load modules+module load matlab/R2022b # load modules
  
-Rscript hello.R # Execute the script+matlab -nodisplay < matlab.m # Execute the script
  
 ``` ```
  
-User can submit the job to the compute queue with the **[sbatch](https://slurm.schedmd.com/sbatch.html)** command. Note that the batch file (and R script and data) must be located at the /home/ disk.+User can submit the job to the compute queue with the **[sbatch](https://slurm.schedmd.com/sbatch.html)** command.
  
 ``` ```
 sbatch submit.sbatch sbatch submit.sbatch
 ``` ```
 +**[sbatch](https://slurm.schedmd.com/sbatch.html)** command submits the job to the job queue and executes the **[bash script](https://en.wikipedia.org/wiki/Bash_(Unix_shell)**. 
  
-User can monitor the progress of the job with the **[squeue](https://slurm.schedmd.com/squeue.html)** command. JOBID is provided by the sbatch commmand when the job is submitted.+Here is an another example on how to analyze variants.
  
 ``` ```
-squeue -j JOBID+#!/bin/bash 
 +#SBATCH --ntasks 1          # Number of task 
 +#SBATCH --time 00:30:00     # Runtime 30min 
 +#SBATCH --mem 2000          # Reserve 2 GB RAM for the job 
 +#SBATCH --partition serial  # Partition to submit 
 + 
 +module load bcftools # load modules 
 + 
 +# filter variants and calculate stats 
 +bcftools filter --include'%QUAL>20' calls.vcf.gz | bcftools stats --output calls_filtered.stats 
 ``` ```
  
-Also while the job is running user can login to executing compute node with the ssh command. When job is over the ssh session is terminated.+Submit the job to computing queue with the **[sbatch](https://slurm.schedmd.com/sbatch.html)** command.
  
 ``` ```
-ssh sampo1+sbatch variants.sbatch
 ``` ```
  
-** Interactive session ** 
  
-User can get an interactive sessions for whatever purpose. For this to be effective free node is more or less required. Following command will open bash session to any free node on the serial parallel for the next 5 minutes.+### Slurm job queue 
 +User can monitor the state of the job with the **[squeue](https://slurm.schedmd.com/squeue.html)** command. JOBID is provided by the sbatch commmand when the job is submitted.
  
 ``` ```
-srun -p serial --pty -t 0-00:05 /bin/bash+squeue -j <jobid>
 ``` ```
  
-** Slurm job efficiency report (seff) and Accounting **+### Output of the job is available in local output file.
  
-SLURM can provide the user with various job statistics. Like memory usage and CPU time. +``` 
-for example with seff (Slurm job effiency report) it is possible to monitor on how efficiency the job was.+less slurm-<jobid>.out 
 +```
  
 +### Scontrol - View or modify Slurm configuration and state.
 +[Scontrol](https://slurm.schedmd.com/scontrol.html) command gives some information about the job, queue (partition) or the compute nodes. This tool can also modify various parameters of submitted job (runtime for example).
 +
 +#### List all compute nodes
 ``` ```
-seff JOBID+scontrol show node
 ``` ```
  
-It is particularly useful to add following line to the end of the sbatch script:+#### List all compute nodes 
 +``` 
 +scontrol show node 
 +```
  
 +#### List all queues/partitions
 ``` ```
-seff $SLURM_JOBID+scontrol show partition
 ``` ```
  
-or if you wish to have more detailed information+#### List information of the given jobid 
 +``` 
 +scontrol show job <jobid> 
 +``` 
 + 
 + 
 +### Slurm job effiency report (seff) 
 +Seff command will give the report of the completed job on how much resources it consumed. The reported information are CPU wall time, job runtime and memory usage.
  
 ``` ```
-# show all own jobs contained in the accounting database +seff <jobid>
-sacct +
-# show specific job +
-sacct -j JOBID +
-# specify fields +
-sacct -j JOBID -o JobName,MaxRSS,MaxVMSize,CPUTime,ConsumedEnergy +
-# show all fields +
-sacct -j JOBID -o ALL+
 ``` ```
 +
 +
guides/slurm/basics.1569225867.txt.gz · Last modified: 29.10.2019 15:10 (external edit)

Page Tools