Skip to content

Speed up ListOperations

Rohit Kothur requested to merge rkothur/improve-list-operations into master

Description

This MR addresses the slow ListOperations query by creating a get_all_operations method in the scheduler, and making the corresponding methods in the DataStore implementations and job.py return Operations instead of just operation names.

Changes proposed in this merge request:

  • Change the logic of the list_operations method in the Operations instance to call a new get_all_operations method in the scheduler.
  • Change the logic of get_all_operations in the aforementioned low-level classes to return Operations directly, rather than operation names.
  • Add some more type hints

Validation

ListOperations should be significantly faster now that it's not doing a SQL query per job.

Issues addressed

Addresses #257 (closed)

Note that there is still a bug where the response is always populated, regardless of whether the operation is done or whether it succeeded; I've raised #258 (closed) for awareness.

Edited by Rohit Kothur

Merge request reports