Skip to content

Add proper context management and remove asyncio

Cal Pratt requested to merge cpratt34/proper-context-managers into master

This MR adds better handling around startup and teardown of components, and removes most of the asyncio usage from the application. Asyncio really needs to be an all in if we want to use it. Without going all in on asyncio, we end up with awkward application setup (such as the non-standard setup of loops) and end up with stalled out code (any blocking code from an async path stops all other async work).

To facilitate this setup, I added a new component call the ContextWorker (which uses a ContextThread created previously) to manage a long lived process. The ContextWorker provides a startup and teardown method to managing the thread, and provides the worker function with a threading.Event to manage cancellations.

All storage objects, indexes, services, and datastores, have been made in context managers, allowing most components to use an ExitStack for managing their startup and teardown actions. Previously these components had stateful startup logic in their constructor with no matching shutdown rules.

Because some components are shared among higher level components, all startup and teardown methods are made such that they can be called multiple times without issue. The first call to start and stop are the only invocations which actually matter. If we were to redo the configuration loading, it would be possible to enter the context for each object once prior to being assembled into the final structure, however, that change would require a much larger rework. This at least moves us in the correct direction.

Tests have been updated to work with the new format, and the cleanup test in particular was rewritten to actually run the cleanup instead of just mocking it out.

Merge request reports