Skip to content

Fix the session being closed too soon

Micaël Bergeron requested to merge 1430-alternative-solution into master

Merge Request Checklist

  • Link to an issue
  • Include the proposed fix or feature
  • Include and update tests for the modified code
  • Include a documentation change
  • Add a CHANGELOG.md entry in the Unreleased section

Once your merge request is complete, please assign it a Meltano maintainer for review cycle. Once the review cycle finished, the reviewer shall approve the merge request so it can be merged by any Meltano maintainer.

Closes #1430 (closed)

This MR is an alternative solution for #1430 (closed), which doesn't rely on an additional session.

The problem comes from the fact that we are running the ELT process inside a context manager that ensure the job will be updated.

The ELT process should then simply update the Job object sent via the elt_context.

The problem

The ELT process ran in a try/except block to handle any errors that can happen in the process and properly exit the CLI invocation. At the end of this block, we were closing the session that we created for the CLI invocation.

The flow was then:

with job.run(session):
  try:
    … elt process
  finally:
    session.close()

That meant that the job.run contextmanager was enable to properly save the Job object because the session had been closed already.

Merge request reports