Update duration at the end of pipeline

What does this MR do?

Moves duration calculation to be done only once at the end of pipeline processing. Currently this is done every one build.

Why was this MR needed?

This is the simplest thing that we can do before properly implementing duration calculation: https://gitlab.com/gitlab-org/gitlab-ce/issues/23523#note_17145614

This is ~Performance improvement that significantly affects: http://performance.gitlab.net/dashboard/db/sidekiq-workers?var-worker=PipelineUpdateWorker%23perform&var-database=Production&from=now-1h&to=now

Does this MR meet the acceptance criteria?

• CHANGELOG entry added
• Documentation created/updated
• API support added
• Tests
  • Added for this feature/bug
  • All builds are passing
• Conform by the merge request performance guides
• Conform by the style guides
• Branch has no merge conflicts with master (if it does - rebase it please)
• Squashed related commits together

cc @godfat

Added ~149423 label

@ayufan I wonder if this does not affect UX . User won't be able to see pipeline duration until all builds and stages are processed. I think it may make sense to move this to Pipeline#update_status or to one of the asynchronous workers. What do you think about it?

Restored source branch update-duration-at-the-end-of-pipeline

@grzesiek

I agree, but the problem is that currently this is also wrongly shown, because it doesn't account "running" builds, but duration is only updated every some time (when build changes state).

The best approximation would be probably Time.now - started_at as this is way more accurate.

Definitely goes the wrong direction from #19805 (moved). But if you can solve #19805 (moved) some other way, such as the suggestion above, then go for it.

@markpundsack

I believe that we should try to solve running one in more efficient way then it's now :)

@ayufan Do we record both when the pipeline was created and when it actually started running (not just pending)?

yes, we do record when it started running, so when the first build became running.

@ayufan Conflict. :(

@Remy im out, could you rebase?

I will try to rebase this branch, but unfortunately I'm not able to pull from gitlab.com at the moment (tried several times so far).

Added 154 commits:

• 2ad6218a...1f949c0a - 153 commits from branch master
• c9108442 - Update duration at the end of pipeline

Compare with previous version

Enabled an automatic merge when the build for c9108442 succeeds

Thanks @grzesiek!

Canceled the automatic merge

Status changed to merged

Mentioned in commit 3c5b9474

Picked into 8-13-stable, will go into 8.13.0-rc4.

Removed ~149423 label

Mentioned in merge request gitlab-com/www-gitlab-com!3412 (merged)

Mentioned in commit eb9d321e