Skip to content

Avoid loading database objects in ES result

Mario de la Ossa requested to merge 10854-elasticsearch_avoid_db into master

What does this MR do?

We already have most of the information we want to show in search results inside Elasticsearch itself, so we should avoid loading models from the database and instead show what we have.

This requires adding "Light" versions of our models, which will live in lib/elasticsearch (feel free to disagree, I'm not certain about this decision)

Required Elasticsearch index changes

  • Milestone iid needs to be indexed as it's used for routing information

Things we stop showing in search results by doing this:

Model Attributes
Project Stars
Issues None!
MRs None!
Milestones None!
Wiki Not touched by this MR
Users Not touched by this MR
Code Not touched by this MR
Commits Not touched by this MR

For more info: #10854 (closed)

Next steps after this

Unfortunately I couldn't get rid of many database calls since we're not storing their results in Elasticsearch. As a follow-up we should decide if the following are worth storing:

  • Projects -> stars, We could also consider caching forks, open MRs, open Issues as right now these are implemented as finders
  • Issues -> full routing info (missing namespace param)
  • Milestones -> full routing info (missing namespace param)
  • MRs -> full routing info (missing namespace param)

In particular, not having the full route for Issues, Milestones, and MRs means that we have to load a Project object for all of them

Does this MR meet the acceptance criteria?

Conformity

Performance and testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Closes #10854 (closed)

Edited by Coung Ngo

Merge request reports