False mismatches when epic gets deleted
Since we enabled syncing epics to work item by default, we see new mismatches especially on start_date
and due_date
. However, when we try to check the epic data, the epic is already deleted in the database. Most of the time the epics got created via the API (mostly REST and some over GraphQL).
My guess is that there are some automations that create and potentially delete epics relatively quickly afterwards. It's hard to find the correlation in the logs, but I was able to find one mismatch that validates the suspicion (see the timestamp of the validation job & the deletion request)
So my guess is the following:
- We create a
EpicWorkItemSync::Diff
instance with theepic
andwork_item
AR objects - We compare attributes on both objects
- In the meantime, the epic gets deleted on the database, however, we still have the AR object
- Then we try to compare the dates, which on the Epic side are stored in the Epic AR (still in memory). However, on the WorkItem, they are stored in
work_items_dates_source
, which we need to query. - So we query the WorkItem::DatesSource but it no longer exists in the database because the epic got deleted
- We therefore have a mismatch on the Epic AR object on
start_date_*
anddue_date_*
Solution
We actually would like to perform a "dirty read" operation, but as this is not possible, we could:
- Query all the work item data at the "same" time as we query the epic data. However even that can lead to mismatches.
- Check if the record still exists in the database every time we perform an attributes match (this sounds expensive though)
- We perform a check at the end of the diff check if the records still exist. Only then we log the mismatch.
I think we should go with option 1 and 3 to prevent false positive warnings.
Open question
The example above explains the case where the data is stored on the Epic AR but we need to query it on the WorkItem side. In the screenshot I however found a mismatch on description
which I can't explain yet.