Update Workflow management
							parent
							
								
									d8056fe707
								
							
						
					
					
						commit
						d733d8aa09
					
				|  | @ -32,17 +32,6 @@ acyclic graph*. This allows two things: | |||
| These two features together provide a solid step towards reproducible simulation | ||||
| work. | ||||
| 
 | ||||
| # GNU Make | ||||
| Make is a program specifically designed to be a build system, i.e. a tool that | ||||
| coordinates the compilation of a program's source code so that an executable or | ||||
| library can be built. Each file of the build process is called a *target* and is | ||||
| the output of some rule. Although it's primary purpose is creating build files, | ||||
| it can easily be made to manage outputs of simulations. While it has the | ||||
| advantage of being installed on virtually every Linux machine used for | ||||
| scientific work, it lacks some features (most notably integration with queue | ||||
| systems) which only make it practical for small cases (although I am sure some | ||||
| shortcomings could be solved with a strong knowledge of Make). | ||||
| 
 | ||||
| # Snakemake | ||||
| Snakemake is a tool written in Python to managed rule-based workflows. The | ||||
| workflow definition is a rather simple text file (usually a `Snakefile`), which | ||||
|  | @ -171,3 +160,30 @@ Here is a list of useful features: | |||
|   the name. | ||||
| - [Parameter space | ||||
|   exploration](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#parameter-space-exploration). | ||||
| 
 | ||||
| # GNU Make | ||||
| Make is a program specifically designed to be a build system, i.e. a tool that | ||||
| coordinates the compilation of a program's source code so that an executable or | ||||
| library can be built. Each file of the build process is called a *target* and is | ||||
| the output of some rule. Although it's primary purpose is creating build files, | ||||
| it can easily be made to manage outputs of simulations. While it has the | ||||
| advantage of being installed on virtually every Linux machine used for | ||||
| scientific work, it lacks some features (most notably integration with queue | ||||
| systems) which only make it practical for small cases (although I am sure some | ||||
| shortcomings could be solved with a strong knowledge of Make). | ||||
| 
 | ||||
| For reference, here is a `Makefile` defining the same rules as the Snakemake example above. | ||||
| 
 | ||||
| ```makefile | ||||
| # One input, one output | ||||
| groups_with_users.txt: /etc/group | ||||
|         cat $< | awk -F ':' '$$4 != "" { print $$1,$$4; }' > $@ | ||||
| 
 | ||||
| # Multiple outputs with grouped targets | ||||
| sorted_groups.txt only_users.txt &: groups_with_users.txt | ||||
|         sort < $< | tee sorted_groups.txt | cut -d ' ' -f 2 > only_users.txt | ||||
| 
 | ||||
| # Rule with pattern | ||||
| start_with_letter_%.txt: groups_with_users.txt | ||||
|         grep '^$*' < $< > $@ | ||||
| ``` | ||||
		Loading…
	
		Reference in New Issue