.md .pdf

repository open issue

Reproducible workflows

Prerequisites:

• docker

• local minikube environment

To-do:

• Have a look at a simplified, but typical physics analysis workflow consisting of two phases

  • first writing the original datasets to a smaller format: producer

  • then running the actual analysis on this smaller format: analyzer (no need to worry about the details of the physics description)

• Have a look how an automated test run of the first part is implemented as a GitHub action workflow

  • What is needed is the container (with the sw installed) where the job is run, and the commands which are passed in a script here

  • you can read the introduction to github actions, but no need to go in details now

• To put together different steps in analysis, workflow engines will be used. This can be done in a kubernetes cluster using argo workflow

  • try the single-step example above using argo in your local minikube environment:

    • you need these two files volumes.yaml and argo-workflow.yaml

    • see the commands needed after run: statements here

Containers