Skip to content

Add refresh timestamp threshold

Jimmy F requested to merge fewer-timestamp-update into master

Before raising this MR, consider whether the following are required, and complete if so:

  • Unit tests

Description

Traditionally, every request to read a blob is paired with a request to write. (To update the blob's access time).
This results in high frequency of timestamp updates during heavy access, and causes the database performance to degrade.

This MR introduces a behavior of refreshing the timestamp only if it hasn't been updated for some amount of time. This amount of time is configurable.

Changes proposed in this merge request:

  • The SQLIndex class has a member variable to store the mentioned threshold hour value.
  • In _bulk_refresh_timestamp(): the accessed_timsetamp of a blob will be refreshed if it's older than the threshold.
  • In get_blob(): it now uses bulk_read_blobs().
  • The select() statement is re-written to sqlalchemy 2.0-compaitble style. And using chaining of where() clauses.

Validation

  • Pass the CI/CD pipeline.
  • Pass the group of unit tests specific to the CAS Index with:
    • tox -e venv -- pytest tests/cas/index/test_index.py
  • Pass lint with:
    • tox -e lint
Edited by Jimmy F

Merge request reports