Skip to content

The Refs Table Grows Too Large

The refs table in Lorry's database overtime grows too large because it tracks detailed information about each ref that it attempts to push.

The refs table schema is as follows:

0|id|INTEGER|1||1
1|job_id|INTEGER|1||0
2|name|TEXT|1||0
3|message|TEXT|1||0
4|successful|BOOLEAN|1|0|0

And some example rows:

696|1007488|refs/heads/alatiera/kernel-6-9-12|! refs/heads/alatiera/kernel-6-9-12:refs/heads/alatiera/kernel-6-9-12     [rejected] (non-fast-forward)|0
697|1007488|refs/heads/alatiera/llvm-bolt|!     refs/heads/alatiera/llvm-bolt:refs/heads/alatiera/llvm-bolt     [rejected] (non-fast-forward)|0
698|1007488|refs/heads/alatiera/rust-lto|!      refs/heads/alatiera/rust-lto:refs/heads/alatiera/rust-lto       [rejected] (non-fast-forward)|0

Because each job tracks every ref the size of this table can grow very large.

A few possibilities to solve this issue are:

  1. Only record failed refs and ignore successful ones
  2. Have a maintenance cadence which deletes ref information after a certain number of days
  3. Only track the latest ref failures
  4. Have a separate table which tracks ref state per job which only contains success/no-success and only the failure message if any exists e.g. force-push-rejected
Edited by Kevin Schoon
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information