Skip to content

Fix ctags

Dmitry Gruzd requested to merge 467121-fix-ctags into master

What does this MR do?

This MR fixes ctags binary so that it starts to actually generate symbols during indexing

Before

time=2024-07-01T17:51:31.694Z level=INFO msg="IndexRepository start" project_id=7 force=true
universal-ctags: Unknown option: --_interactive
time=2024-07-01T17:51:32.416Z level=INFO msg="ignoring universal:/usr/bin/universal-ctags or scip: error: write |1: file already closed"
time=2024-07-01T17:51:32.417Z level=INFO msg="symbol analysis finished for shard statistics: duration=0s symbols=0 bytes=39"
time=2024-07-01T17:51:32.437Z level=INFO msg="finished shard /data/index/7_v16.00000.zoekt: 582787 index bytes (overhead 3.5), 42 files processed "
time=2024-07-01T17:51:32.437Z level=INFO msg="IndexRepository end" project_id=7 force=true

After

time=2024-07-01T18:18:02.636Z level=INFO msg="IndexRepository start" project_id=7 force=true
time=2024-07-01T18:18:03.498Z level=INFO msg="symbol analysis finished for shard statistics: duration=0s symbols=866 bytes=165163"
time=2024-07-01T18:18:03.516Z level=INFO msg="finished shard /data/index/7_v16.00000.zoekt: 610141 index bytes (overhead 3.7), 42 files processed "
time=2024-07-01T18:18:03.517Z level=INFO msg="IndexRepository end" project_id=7 force=true

image

Steps to reproduce

  1. Build the CNG image docker build -t cng_indexer .
  2. Enable Zoekt in GDK
  3. Stop the indexer
    gdk stop gitlab-zoekt-indexer-development
  4. Update gitaly/praefect.config.toml to include
    listen_addr = "0.0.0.0:2305"
  5. Restart praefect
    gdk restart praefect
  6. Start the indexer container
    docker run -p 6080:6065 -v $GDK_DIR/zoekt-data/development/index:/data/index --rm -it cng_indexer
  7. Update ee/lib/gitlab/search/zoekt/client.rb
    diff --git a/ee/lib/gitlab/search/zoekt/client.rb b/ee/lib/gitlab/search/zoekt/client.rb
    index 5e7755bd5dd4..9c90d5d220cf 100644
    --- a/ee/lib/gitlab/search/zoekt/client.rb
    +++ b/ee/lib/gitlab/search/zoekt/client.rb
    @@ -161,7 +161,7 @@ def indexing_payload(project, force:, callback_payload:)
              repository_storage = project.repository_storage
              connection_info = Gitlab::GitalyClient.connection_data(repository_storage)
              repository_path = "#{project.repository.disk_path}.git"
    -          address = connection_info['address']
    +          address = 'tcp://host.docker.internal:2305'
    
              # This code is needed to support relative unix: connection strings. For example, specs
              if address.match?(%r{\Aunix:[^/.]})
    
  8. Open rails console and execute
    Zoekt::IndexerWorker.new.perform(7, { 'force' => true })
  9. You should see ctags being generated. You can also test it by performing a Zoekt search with sym: syntax

Related issues

gitlab-org/gitlab#467121 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Integration tests added to GitLab QA
  • The impact any change in container size has should be evaluated
  • New dependencies are managed with dependencies.io
Edited by Jason Plum

Merge request reports