Skip to content

Auto-Configure the SQLDataStore for SQLite with correct args

Due to the limitations of SQLite and the features we use (e.g. ORM), it looks like we have to take into account a few things:

  • Use a StaticPool so that all threads share the same connection (helps with race conditions and allows us to use ORM Sessions)
  • Disable check_same_thread so that SQLite doesn't complain about us trying to use it from different threads — since we're using the StaticPool in combination, we won't have race conditions w/concurrent sqlite access.
  • Use a NamedTemporaryFile instead of just shared memory: this will help us down the line allow a different Indexed CAS database and State DataStore database. — also, couldn't get the named shared memory database to work (with exotic connection strings like: sqlite:///file:bgd-inmem?mode=memory&cache=shared&uri=true)

This MR updates the SQLDataStore to:

  • Automatically configure the above options for SQLite
  • Disallow using in-memory sqlite if explicitly asked for (since it can be problematic with our multi-threaded model)
  • Updates docs

This MR also updates the migration env.py to receive the engine exactly as configured by the user in SQLDataStore (instead of only using the connection string and use the default settings for everything else).

See the SQLAlchemy docs for more info.

Edited by Marios Hadjimichael

Merge request reports