Query API's performance degrades over time and needs periodic restart
Observed behavior
Without any exceptions thrown the query api appears to slow down to the point where requests time-out and health checks fail due to these time-outs.
Expected behaviour
The api should have a linearly proportionate response time independently of how long it is running. If that is not achievable in the short term, it should at least be able to auto-restart when its performance degrades. Suggesting to look at Docker's health check facilities.
How urgent
The API's stability is a top priority. The issue though is not constantly ongoing, so some time can be bought with manual intervention for the time being.
Definition of ready
-
Product owner: @mrittman -
Tech lead: @ppandis -
Service:: label applied -
Definition of done updated -
Acceptance testing plan: -
Weight applied
Definition of done
-
Unit tests identified, implemented, and passing -
Code reviewed -
Available for acceptance testing via a staging URL, or otherwise -
Consider any impacts to current or future architecture/infrastructure, and update specifications and documentation as needed -
Knowledge base reviewed and updated -
Public documentation reviewed and updated -
Acceptance criteria met -
The API's performance is directly proportional to the workload presented or the API restarts when its performance degrade's
-
-
Acceptance testing passed -
Deployed to production