Skip to content

POC - Override due dates for sample data

What does this MR do?

In order to precalculate and override due_dates for https://gitlab.com/gitlab-com/www-gitlab-com/-/issues as described in gitlab-com/www-gitlab-com#7397 (comment 416758822), we can do the precalculation of median date in pre-import work, where we could read both issues.ndjson and milestones.ndjson. (This will work only for ndjson exports). We are precalculating the dynamic date (median date from all issues and milestones), and then we continue calculating new values on the fly while importing each relation.

This is explained in 1st approach here

Pros

  • It's simple. The JSON is not complex and nested, we only need to read issues.ndjson and milestones.ndjson, which contains a really simple structure. It's easy and simple to extract all due_dates and calculate the median value.
  • It's performant. We update/override actual due_dates values as we import them. So actual objects are persisted with precalculated due dates. No additional reads/writes to the database.

Cons

  • NDJSON reader was designed to consume relations. Once a relation is consumed, you cannot consume it again. The whole idea around NDJSON was to consume whole import by reading it in one go. Now we need to read all issues and milestones in advance, in order to calculate the median date, and then we must read each relation (issue/milestone) again when we import/restore them.

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Grant Young

Merge request reports