Overview

Snakemake 7.0, released on October 15, 2021, improves cluster execution and the checkpoint system for bioinformatics workflows.

Main Features

Improved cluster execution

Cluster execution benefits from a new executor system with configured profiles for Slurm, SGE, and other job schedulers.

python 
# Snakefile
rule align:
    input:
        'data/{sample}.fastq'
    output:
        'results/{sample}.bam'
    threads: 8
    resources:
        mem_mb=16000
    shell:
        'bwa mem -t {threads} ref.fa {input} | '
        'samtools sort -o {output}'

# Run on Slurm cluster:
# snakemake --executor slurm --jobs 10

Checkpoints

Checkpoints allow re-evaluating the DAG after a rule executes, useful when the number of output files is only known at runtime.

python 
# Snakefile with checkpoint
checkpoint split:
    input:
        'data/large_file.csv'
    output:
        directory('data/chunks/')
    shell:
        'mkdir -p {output} && split -l 1000 {input} {output}/part_'

def aggregate_inputs(wildcards):
    """Determine files after checkpoint."""
    import glob
    checkpoint_output = checkpoints.split.get(**wildcards).output[0]
    return glob.glob(f'{checkpoint_output}/part_*')

rule analyze:
    input: aggregate_inputs
    output: 'results/analysis.txt'
    shell: 'cat {input} | wc -l > {output}'

Sources