lfrerot
/
good_simulation_practices
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Workflow management

Lucas FRÉROT 2024-12-15 16:24:02 +00:00
parent e7008f5f61
commit 4630147117
1 changed files with 12 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
Workflow-management.md

@ -123,7 +123,7 @@ re-run the last step.
The rule syntax is rather straight-forward: each rule has a list of inputs and The rule syntax is rather straight-forward: each rule has a list of inputs and
outputs (which are numbered from `0` to `N` by default, and can be named). The outputs (which are numbered from `0` to `N` by default, and can be named). The
`shell` directive specifies that we want to run a shell command. This is the `shell` directive specifies that we want to run a shell command. This is the
most flexible option. Alternatively one can use the `run` directive and write most flexible option. Alternatively, one can use the `run` directive and write
inline python code directly in the `Snakefile`, the `script` directive, which inline python code directly in the `Snakefile`, the `script` directive, which
specifies the name of a Python (or another language) specifies the name of a Python (or another language)
[script](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#external-scripts) [script](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#external-scripts)
@ -131,7 +131,7 @@ to be run (Snakemake creates a context for this script which allows it to access
the input and output objects), or finally the [`notebook` the input and output objects), or finally the [`notebook`
directive](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#jupyter-notebook-integration), directive](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#jupyter-notebook-integration),
similar to the `script` directive, for which Snakemake allows interactive similar to the `script` directive, for which Snakemake allows interactive
execution (useful for postprocessing/data exploration). execution (useful for post-processing/data exploration).
Reading the Reading the
[documentation](https://snakemake.readthedocs.io/en/stable/index.html) is highly [documentation](https://snakemake.readthedocs.io/en/stable/index.html) is highly
@ -165,7 +165,7 @@ Here is a list of useful features:
Make is a program specifically designed to be a build system, i.e. a tool that Make is a program specifically designed to be a build system, i.e. a tool that
coordinates the compilation of a program's source code so that an executable or coordinates the compilation of a program's source code so that an executable or
library can be built. Each file of the build process is called a *target* and is library can be built. Each file of the build process is called a *target* and is
the output of some rule. Although it's primary purpose is creating build files, the output of some rule. Although its primary purpose is creating build files,
it can easily be made to manage outputs of simulations. While it has the it can easily be made to manage outputs of simulations. While it has the
advantage of being installed on virtually every Linux machine used for advantage of being installed on virtually every Linux machine used for
scientific work, it lacks some features (most notably integration with queue scientific work, it lacks some features (most notably integration with queue
@ -188,4 +188,12 @@ sorted_groups.txt only_users.txt &: groups_with_users.txt
# Rule with pattern # Rule with pattern
start_with_letter_%.txt: groups_with_users.txt start_with_letter_%.txt: groups_with_users.txt
grep '^$*' < $< > $@ grep '^$*' < $< > $@
``` ```
Here are [documentation](https://www.gnu.org/software/make/manual/html_node/index.html) pages for interesting features used in the example:
- [Rule syntax](https://www.gnu.org/software/make/manual/html_node/Rule-Syntax.html)
- [Wildcards](https://www.gnu.org/software/make/manual/html_node/Wildcards.html) are semantically different from wildcards in Snakemake
- [Pattern rules](https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html) correspond to wildcards in Snakemake
- [Automatic variables](https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html) correspond to the symbols `$@`, `$<` and `$*` in the example
- [Grouped targets](https://www.gnu.org/software/make/manual/html_node/Multiple-Targets.html)