Speed up ListOperations
Description
This MR addresses the slow ListOperations query by creating a get_all_operations
method in the scheduler, and making the corresponding methods in the DataStore
implementations and job.py
return Operations instead of just operation names.
Changes proposed in this merge request:
- Change the logic of the
list_operations
method in the Operations instance to call a newget_all_operations
method in the scheduler. - Change the logic of
get_all_operations
in the aforementioned low-level classes to return Operations directly, rather than operation names. - Add some more type hints
Validation
ListOperations should be significantly faster now that it's not doing a SQL query per job.
Issues addressed
Addresses #257 (closed)
Note that there is still a bug where the response is always populated, regardless of whether the operation is done or whether it succeeded; I've raised #258 (closed) for awareness.
Edited by Rohit Kothur