Maybe support non-path basf2 configuration in gbasf2/grid batch

[meliache]

I just sent the basf2 path to the grid, but sometimes the user might want to call some basf2 configuration functions which change the internal state of basf2, but are not stored in the path, and they might not be available via CLI arguments. Examples include

• basf2 aliases (stored in the variable manager)
• testing payloads
• conditions
• ... more?
  In principle I could let the user add all these things via additional settings. E.g. let the user provide a dictionary with all the aliases and a list with the testing payloads, then in the steering file wrapper I can add the aliases and the testing payloads. But it's hard to anticipate everything the user might want and it might bloat my code.

It would be nice if the user could provide a function or a file in which all basf2 configuration functions, that the user needs, are called. I'm not sure however how to call that from the steering file wrapper. I could send an additional basf2 configuration file to the grid, but not sure how I could import that configuration from the steering file wrapper, without forcing the user to supply a specific function name.

Another possiblity would be to have the user supply his own steering file template. Code-wise this seems the simplest solution, so I already did a test-implementation on a branch, but it is dangerous and would require careful documentation, here is my first attempt

But I am not sure I like this solution, it's not something normal users should have to use for some simple configuration.

I would be happy for some suggestions if you have any ideas how to support basf2 configuration without adding much complexity and in a simple way for the user. I would be happy to do implement that myself.

[nils-braun]

I will look into this soon, but are you aware of the pickable_basf2 module that tkeck wrote (framework/scripts). I do not know if this is still working or in use, but I guess if something is already supported by basf2, we should use that.

[meliache]

Niceeeee, looks very good. Wasn't aware of when writing this, might have heard. Will definetly test that. Might have found that earlier when reading more carefully the code of pickle_path.py, because the function make_code_pickable() mentions that.

I didn't create this issue for necessarily for you to invest time looking into that, just wanted to have that as a TODO basically assigned to myself, but where I would be happy for outward input. This was one such example. But it's not urgent.

[meliache]

I looked more carefully at the code of pickable_basf2.py (the link ist for a gist version in case you don't have a basf2 copy anymore). Seems that it is geared towards being used when serializing/executing paths from the commandline with basf2 --dump-path / --execute-path, as it works with a changed process method which writes the state into the pickle file.

I am just using get_path_from_file and write_path_to_file, but I think I might be able use the ideas from that file. E.g. use unittest.mock and maybe also a Basf2StateRecorder class to record all calls to basf2 in the create_path() function body.

I don't think I should import pickable_basf2.py. First of all, I don't want to replace basf with the Basf2StateRecorder for the entire b2luigi worker process, and also it does not really seem to work to import parts for it, at least I get an import error when running

from pickable_basf2 import BASF2StateRecorder

[meliache]

Conditions have been added via #77. I discussed an idea for a more general solution in this comment