Extending QUEENS drivers for more general workflows

[gilrrei]

This issue was originally created by @gilrrei on gitlab.lrz.de on 2024-05-12.

Motivation and Context

For a project I have to nest various workflows and run them on the cluster. During the hackmania the following issues w.r.t. drivers appeared

• Multiple input files Currently we have the possibility to copy arbitrarily amount of files to experiment_dir. This could be 4C solver settings or similar. To ease the usage of these paths I suggest to switch files_to_copy from a list to a dict (yes yes I implemented it as list) and implement a method get_copied_file(self, file_name) which returns the path of the file (or folder) from experiment_dir. This generalizes the workflow from one to multiple input files/folders and eases accessing the files. Currently, if you change of the files in the list and it is later needed for creating input files, the name of the file in the code also needs to be changed. → 754
• Unify drivers We have some redundancy in MpiDriver and JobscriptDriver. This leads to additional effort when creating new drivers for a specific purpose, as the run methods are different. Somehow we should generalize this. → 737
• Python driver We should revive a python driver. This is nice to run post-processing on existing simulation data. → 748
• Sync other python projects Sometimes one has to work on multiple python packages at once. Here we should generalize the queens syncing capabilities to not only sync QUEENS but also other packages automatically.

Current Behavior

Possible Solution

Related Issues

• Blocks
• Is blocked by
• Follows
• Precedes
• Related to 497
• Part of
• Composed of

Additional Information

Interested Parties