I'm working on adding features to the snakemake aws batch executor plugin. The existing plugin supports execution on AWS Batch by dynamically creating job definitions based on rule resource configuration, but was missing support for features like using different containers for different rules, consumable resources, secrets, etc. Two approaches:
1) https://github.com/radusuciu/snakemake-executor-plugin-aws-b... (my fork). Just add the features to the batch job building code
2) https://github.com/radusuciu/snakemake-executor-plugin-aws-b.... This is more experimental and not yet fully working. I wanted to try a few things. a) can we rely on existing job definitions (managed through IaC instead). b) can we implement a fire-and-forget model where the main snakemake process runs on Batch as well? c) Can we slim down the snakemake container by stripping off unnecessary features.
1) https://github.com/radusuciu/snakemake-executor-plugin-aws-b... (my fork). Just add the features to the batch job building code 2) https://github.com/radusuciu/snakemake-executor-plugin-aws-b.... This is more experimental and not yet fully working. I wanted to try a few things. a) can we rely on existing job definitions (managed through IaC instead). b) can we implement a fire-and-forget model where the main snakemake process runs on Batch as well? c) Can we slim down the snakemake container by stripping off unnecessary features.