Skip to content

Generate sitemap routes statically

Francisco Javier López (ex-Gitlab) requested to merge fj-generate-sitemap-routes-statically into master

What does this MR do?

In this MR we fix two things. The first one we want is to improve performance of the sitemap generation by creating the urls manually instead of using the Rails route helpers. The problem is, what if a route change? The sitemap urls will be invalid. In order to solve this, we have changed the specs and use the Rails route helpers there to ensure that the statically routes are correct. Therefore, if a route changes we will detect it.

The second thing we changed in this MR is the availability of features. We didn't consider that features like issues or merge requests could be disabled, and they will return a 404 to the crawler if they are. Moreover, we need to check not only that it is enabled but visible to the user.

In local, with 50 groups and 30 projects, the generation time was improved by 200ms. Therefore, if we extrapolate this to the number of groups and projects of the gitlab-org group, we think this time will improve considerably.

Does this MR meet the acceptance criteria?

Conformity

Refs #270051 (closed)

Edited by Francisco Javier López (ex-Gitlab)

Merge request reports