Some functionality was orphaned and forgotten in the last-minute changes to core gmxapi infrastructure as the Python package was merged to GROMACS 2020. Issue #3145 remains to be addressed, but we should restore some of the gmxapi 0.0.7 functionality that was lost in gmxapi 0.1. A logic error was introduced as `gmxapi::System` was deprecated, in which `IRestraintPotential` providers added to the system are not attached to the simulation that is launched. This was possible because of some confusing redundancy that was introduced, in which a container was duplicated in the `gmxapi::System` and `gmxpy::PyContext` proxy, but roles were not completely updated. ## Background A working updated protocol is demonstrated in the C++ tests, illustrated in the following sequence diagram. ```plantuml 'https://plantuml.com/sequence-diagram 'autonumber autoactivate on client -> libgmxapi: gmxapi::fromTprFile() libgmxapi -> "gmxapi::System" **: create return "gmxapi::System" client -> libgmxapi: gmxapi::createContext() libgmxapi -> "gmxapi::Context" **: create return "gmxapi::Context" client -> "gmxapi::Context": setMDArgs() return client -> plugin: create plugin -> restraint **: create return restraint client -> "gmxapi::System": System.launch(gmxapi::Context) "gmxapi::System" -> "gmxapi::Context": launch(systemimpl.workflow_) return session loop foreach member in systemIml.spec_->getModules() "gmxapi::System" -> libgmxapi: addSessionRestraint(session, member) note right spec is empty! end note return end return "gmxapi::Session" client -> libgmxapi: gmxapi::addSessionRestraint(session, restraint) libgmxapi -> sessionImpl: addRestraint(restraint) sessionImpl -> "gmx::MdRunner": addPotential(restraint) return return success return success client -> "gmxapi::Session": run() "gmxapi::Session" -> "gmx::MdRunner": run() return return ``` It took an extra few years to enable automated testing of Python code in the GROMACS CI pipelines, during which time development and test coverage languished. The broken protocol is illustrated in the following sequence diagram. ```plantuml @startuml 'https://plantuml.com/sequence-diagram 'autonumber autoactivate on context.py -> _gmxapi: Context() _gmxapi -> "gmxpy::PyContext" **: create "gmxpy::PyContext" -> "gmxapi::Context" **: gmxapi::createContext() return ... context.py -> plugin: _builder.add_subscriber(mdbuilder) deactivate context.py -> plugin: _builder.build(graph) plugin -> restraint **: create plugin -> context.py: mdbuilder.potential.append(restraint) deactivate deactivate plugin ... context.py -> context.py: mdbuilder.build(graph) 'in MD builder-> launch context.py -> _gmxapi: from_tpr() _gmxapi -> "gmxapi::System" **: create return system context.py -> _gmxapi: setMDArgs() _gmxapi -> "gmxpy::PyContext": setMDArgs() "gmxpy::PyContext" -> "gmxapi::Context": setMDArgs() deactivate deactivate deactivate _gmxapi context.py -> _gmxapi: add_mdmodule(plugin) _gmxapi -> "gmxpy::PyContext": addMDModule(restraint) "gmxpy::PyContext" -> "gmxpy::PyContext": getSpec() return this->workNodes_ "gmxpy::PyContext" -> restraint: bind(pycontext.workspec) restraint -> "gmxpy::PyContext": gmxapi::MDWorkSpec::addModule(module) note right restraint added to PyContext member. end note deactivate deactivate deactivate deactivate context.py -> _gmxapi: system.launch(context) _gmxapi -> "gmxpy::PyContext": get() return gmxapi::Context _gmxapi -> "gmxapi::System": System.launch(gmxapi::Context) "gmxapi::System" -> "gmxapi::Context": launch(systemimpl.workflow_) return session loop foreach member in systemIml.spec_->getModules() "gmxapi::System" -> gmxapi: addSessionRestraint(session, member) deactivate note right spec is empty! end note end return gmxapi::Session return _gmxapi.MDSession return ' in session launch context.py -> _gmxapi: _gmxapi.MDSession.run() _gmxapi -> "gmxapi::Session": run() ... @enduml ``` ## Proposal * Make minimal changes to the gmxapi interfaces related to `gmxapi::System` and `gmxapi::Context` at this time. * Update the `gmxapi._gmxapi.add_mdmodule()` function in the Python bindings to allow the target of the plugin `bind()` behavior to target the correct instance of the container. * Add a test to `python_packaging/sample_restraint` to confirm that IRestraintPotental code is actually called in GROMACS 2020+ / gmxapi 0.1+. * Follow up in issue #4079 to migrate to using a `SimulationInput` and expanded `SimulationContext` interface, and remove `gmxapi::System`, `gmxapi::Workflow`, and `gmxapi::WorkSpec` The proposed patch for GROMACS 2020, 2021, and 2022 is illustrated in the following sequence diagram. ```plantuml @startuml 'https://plantuml.com/sequence-diagram 'autonumber autoactivate on context.py -> _gmxapi: Context() _gmxapi -> "gmxpy::PyContext" **: create "gmxpy::PyContext" -> "gmxapi::Context" **: gmxapi::createContext() return ... context.py -> plugin: _builder.add_subscriber(mdbuilder) deactivate context.py -> plugin: _builder.build(graph) plugin -> restraint **: create plugin -> context.py: mdbuilder.potential.append(restraint) deactivate deactivate plugin ... context.py -> context.py: mdbuilder.build(graph) 'in MD builder-> launch context.py -> _gmxapi: from_tpr() _gmxapi -> "gmxapi::System" **: create return system context.py -> _gmxapi: setMDArgs() _gmxapi -> "gmxpy::PyContext": setMDArgs() "gmxpy::PyContext" -> "gmxapi::Context": setMDArgs() deactivate deactivate deactivate _gmxapi context.py -> _gmxapi: add_mdmodule(plugin) _gmxapi -> "gmxpy::PyContext": addMDModule(restraint) "gmxpy::PyContext" -> "gmxpy::PyContext": getSpec() return this->workNodes_ "gmxpy::PyContext" -> restraint: bind(pycontext.workspec) restraint -> "gmxpy::PyContext": gmxapi::MDWorkSpec::addModule(module) note right restraint added to PyContext member. end note deactivate deactivate deactivate deactivate context.py -> _gmxapi: system.launch(context) _gmxapi -> gmxapi: getWork(system) note right We now only use gmxapi::System to hold the TPR filename. We can replace gmxapi::System with gmxapi::SimulationInput. end note gmxapi -> "gmxapi::System": systemimpl.workflow_ return return gmxapi::Workflow _gmxapi -> "gmxpy::PyContext": launch(work) "gmxpy::PyContext" -> gmxapi: launchSession(context, work) gmxapi -> "gmxapi::Context": launch(work) note right "gmxapi::Context" Convert filename to SimulationInput. (Should be done before entering ContextImpl::launch()) end note return return "gmxapi::Session" "gmxpy::PyContext" -> gmxapi: this->workNodes_->getModules() return restraints loop foreach restraint "gmxpy::PyContext" -> gmxapi: addSessionRestraint(session, restraint) deactivate end return gmxapi::Session return _gmxapi::MDSession return session ' in session launch context.py -> _gmxapi: _gmxapi.MDSession.run() _gmxapi -> "gmxapi::Session": run() ... @enduml ``` ## Relates to See #3145 and #4079