Skip to content
Snippets Groups Projects

Update thresholds for code suggestions

Merged Jan Provaznik requested to merge jp-split-code-suggestions into master
1 unresolved thread

Related to gitlab-org/gitlab#425095 (comment 1739978289)

$ ./test-dashboard.sh ./ai-gateway/main.dashboard.jsonnet
Installed https://dashboards.gitlab.net/dashboard/snapshot/T6c67Xc0LNIIIvgSkTmzf0o4qaQkWJ0Z - ai-gateway: Overview

Update thresholds for code suggestions

  • increase default threashol for code suggestions (as code generations take longer time)
  • use separate apdex for completions and generations

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Jan Provaznik
  • Jan Provaznik requested review from @cfeick

    requested review from @cfeick

  • Chance Feick
  • Jan Provaznik added 1 commit

    added 1 commit

    • d1fd346f - Autogenerate rules and fix dashboard

    Compare with previous version

  • Author Developer

    Thanks @cfeick, I've updated the MR, could you please take a look?

  • Jan Provaznik requested review from @cfeick

    requested review from @cfeick

  • Chance Feick resolved all threads

    resolved all threads

  • Chance Feick
  • Jan Provaznik added 1 commit

    added 1 commit

    • e53435c5 - Add trafficCessationAlertConfig based on feedback

    Compare with previous version

  • Jan Provaznik requested review from @cfeick

    requested review from @cfeick

  • Chance Feick approved this merge request

    approved this merge request

  • Jan Provaznik requested review from @reprazent

    requested review from @reprazent

  • 40 41
    41 42 apdex: histogramApdex(
    42 43 histogram='http_request_duration_seconds_bucket',
    43 selector=baseSelector { status: { noneOf: ['4xx', '5xx'] } },
    44 satisfiedThreshold=2.5,
    44 selector=baseSelector { status: { noneOf: ['4xx', '5xx'] }, handler: { noneOf: ['/v2/code/completions', '/v2/completions', '/v2/code/generations'] } },
    • Extracting this stuff out per route works okay in the short term, ideally we'd be moving this into the application in the form of the Application SLIs, like we have for the Rails application. That way, the application specifies what is "fast enough" and we don't need to create separate SLIs for different types of routes.

      Ideally, this stuff would move to labkit, and we'd have a labkit-python version to support the AI-gateway.

      But that's out of scope of this, obviously... I'll create issues to discuss this further in Scalability. I think there's some overlap between Observability and Practices for this (cc @cfeick, @abrandl as we've discussed this in the past).

    • Author Developer

      That way, the application specifies what is "fast enough" and we don't need to create separate SLIs for different types of routes.

      @reprazent good point, it would be better to do it that way 👍

    • Please register or sign in to reply
  • Bob Van Landuyt resolved all threads

    resolved all threads

  • Bob Van Landuyt approved this merge request

    approved this merge request

  • Thanks @cfeick, @jprovaznik, I had a thought. But nothing that should block here. So I'll merge.

    Please keep in mind that for the error budget for stage groups, it will take 28d for the metrics with the old SLIs to roll out of the budget.

  • Bob Van Landuyt added this merge request to the merge train at position 2

    added this merge request to the merge train at position 2

  • Bob Van Landuyt mentioned in commit 5ef2da26

    mentioned in commit 5ef2da26

  • Contributor

    A pipeline is running on a mirror related to this merge request.

    Status: starting

    https://ops.gitlab.net/gitlab-com/runbooks/-/pipelines/2815157

  • Contributor

    🎉 This MR is included in version 2.358.2 🎉

    The release is available on GitLab release.

    Your semantic-release bot 📦 🚀

  • Please register or sign in to reply
    Loading