Skip to content

WIP: Use scoped session

Marios Hadjimichael requested to merge marios/scoped-session into master

This MR makes use of the SQLAlchemy scoped_session to ensure we only get one per grpc request.

TODO:

  1. Figure out a way to identify the scope (e.g. grpc request UUID).
  • the current MR is BROKEN because we get one scoped_session per THREAD (and the same thread handles multiple requests).
  1. Do the equivalent work for CAS, if needed.

Strategies to identify scope:

1. Generate a UUID with each grpc request at the GRPC handling level.

1.1. Pass it to the data_store using threading.local() (a bit messy, but won't need to pass it to all methods)

1.2. Pass the UUID down to all the methods involved with handling the request (maybe using a special RequestContext object?) — couples the implementation a bit, must update a lot of code for that

2. Introduce the concept of SchedulerSessions, and have whoever is calling the scheduler first in the chain of function calls from grpc request handling to create "open a scheduler session" (e.g. using a decorator?)

2.1. Have the SchedulerSession call data_store.begin_session() and generate a UUID, stored in threading.local() (easier implementation, threading.local() data)

2.2. Generate a UUID and pass it through all the function calls made to the scheduler down to the data_store. (more work/refactoring, easier traceability of the UUID)

3. Event call-backs for (when we start?) and end handling an rpc request?

3.1. using grpc interceptors

3.2. Figuring out if that is possible with ThreadPool.submit() and how to tell grpc to do that when submitting

Also, a few classes are using the data_store directly, e.g. class Job — we should probably refactor it to make the data_store a function arg and pass it in as we call the function (to decouple it).

Related to: #221 (closed)

Edited by Marios Hadjimichael

Merge request reports