AI Impact Analytics - Adding Gitlab Duo Enterprise enforcement

Overview

In the GA, the pricing model for the AI Impact was that feature is packaged as part of Duo Enterprise, and will be available for Ultimate for a limited time, and will require Ultimate and Duo Enterprise after limited free access period.

Problem

After we introduce Duo Enterprise, we need to ensure the AI Impact will be available only for Ultimate with Duo Enterprise add on.

Proposal

• Adding Gitlab Duo Enterprise enforcement for AI Impact behind a feature flag.
• Show AI impact analytics only for users assigned to an add-on seat

Related also to https://gitlab.com/gitlab-org/fulfillment/meta/-/work_items/1943+

marked this issue as related to #456078 (closed)

@blabuschagne following our chat, here is a new issue for the Duo Ent enforcement.

added groupoptimize typefeature labels

added devopsplan sectiondev labels

changed the description

changed milestone to %Next 1-3 releases

added to epic &12978

added priority1 label

added backend workflowplanning breakdown labels

cc @gitlab-org/plan-stage/optimize-group/engineers for breakdown (tagging frontend here too in case we'd need to gate specific components too)

added estimationneeded label

set weight to 3

added estimationcompleted label and removed estimationneeded label

added workflowscheduling label and removed workflowplanning breakdown label

changed the description

Based on the timeline in https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/1965#note_2049687648 I'm going to pull this into %17.4

We are still clarifying the feature flag details so whoever picks this up will need to hold off merging anything everything is confirmed - but we should be able to get a start on this already.

cc @gitlab-org/plan-stage/optimize-group/engineers/backend

changed milestone to %17.4

added workflowready for development label and removed workflowscheduling label

assigned to @felipe_artur

mentioned in merge request !163036 (merged)

This feature issue does not have the documentation label. Please add it if appropriate, because documentation is one of the aspects of our MR acceptance checklist.

mentioned in issue gitlab-org/quality/triage-reports#19284 (closed)

added workflowin dev label and removed workflowready for development label

@mkaeppler @lwanko Can you give some advice about this issue, please? For context, we want to have AI impact chart available only when duo enterprise add-on is present.

1. Should we use CloudConnector to check if ai impact dashboard is part of duo enterprise add-on?
2. If the first is true, should we add the ai_impact to the configuration as a separate service or a unit primitive of code_suggestions service? I believe we should add it as a separate service in case we add additional AI metrics like duo chat unique users.
3. Should we use cut-off dates instead of a feature flag, similar to what we do to check duo chat availability on global policy? I believe we could avoid backports using this approach as the authorization token is issued by customers dot.

cc @blabuschagne

Just based on the issue description, maybe yes, but I don't have enough context to say for sure. How is this feature delivered, is this implemented entirely in Rails? Cloud Connector primarily solves the problem of delivering features from GitLab-operated backends like the AI gateway.

cc @rogerwoo @sguyon @pjphillips @nmilojevic1

@felipe_artur, AFAIK add-on purchases and available add-ons are registered on the Rails backend (PG DB) so we might have policy files available. @bhrai could you advise here?

@felipe_artur @ahegyi

In the Rails backend (PG DB), we have information on if a user has an assigned seat to a particular add-on, like Duo Pro or Duo Enterprise .

To check if the user has access to particular features that gets unlocked by that add-on, it should be part of the config file cloud_connector.yml . This is then used in the policy file (code_suggestion_enabled_for_user) to build new policy (acess_code_suggestion).

Like Matthias said, if the feature is something completely in Rails, and doesn't involve AI Gateway, may be we can create a new policy based on similar rules. However, I guess if it is AI related then it would use AI Gateway.

Since the customer needs to bring their clickhouse (correct me if I am wrong here), this should not be considered a cloud feature, as we don't have a trusted backend that could leverage our authentication mechanism.

The team can implement the check for add-on purchases and even the seat assignments in Rails monolith, Theoretically, customer can workaround this, but since this is just collecting data for DuoEnterprise features... it would not benefit anyone who didn't already purchase the add-on.

Also, I believe this feature is already shipped in 17.3 with no Enterprise enforcement. This means that customers on older Self Managed instances will still have access to the feature until they upgrade to the version where the "cut off" and Duo Enterprise enforcement are implemented

The team could reuse the CloudConnector::AvailableServices.find_by_name(:ai_impact).free_access? (that would check the cut_off date) and CloudConnector::AvailableServices.find_by_name(:ai_impact).allowed_for(current_user) that would enforce Enterprise purchase and seat assignments.

As Bishwa suggested, in order for this to work the service should be added to CustomersDot config file cloud_connector.yml.

I would strongly vote against it! Because this feature has nothing to do with CloudConnector, and CustomersDot, and it doesn't even USE cloud connector service JWT tokens for authentication.

If the cut-off date is already known, I would suggest to add something like this:

AI_IMPACT_CUT_OFF_DATE = '2024-12-17T00:00:00Z'

def ai_impact_available?
  If Time.parse(AI_IMPACT_CUT_OFF_DATE).utc.past?
    GitlabSubscriptions::AddOnPurchase.by_add_on_name(:duo_enterprise)
      .active.assigned_to_user(current_user)
  else
    true
  end
end

Yes, I would vote for using a manual approach to this. Going through Cloud Connector abstractions that weren't built for features that don't use cloud services is not a good idea, as this may have unintended side effects later on.

Simply speaking, a "Cloud Connector feature" is something that requires dialing cloud.gitlab.com, which does not seem to be the case here, so we shouldn't shoehorn solutions.

Thank you all for the quick responses.
@nmilojevic1 Thanks for the snippet.

Since the customer needs to bring their clickhouse (correct me if I am wrong here), this should not be considered a cloud feature

I think you are right. I found a related discussion that enforces this outcome.

Yes, I would vote for using a manual approach to this.

Yes, makes sense. Also, @blabuschagne pointed out that we want to test this new behavior as soon as possible before the cut-off date, so a feature flag should be used.

Simply speaking, a "Cloud Connector feature" is something that requires dialing cloud.gitlab.com, which does not seem to be the case here, so we shouldn't shoehorn solutions.

I was a bit confused because AI Impact is dependent on features that use Cloud Connector, like code suggestions, so I assumed it was supposed to be a unit primitive of these other features. As the add-on subscription data is present on PG I guess we should not overcomplicate the authorization.

Hi @felipe_artur! Great questions! As I see, my colleagues have answered them already.

If I understand correctly, we don't need to go here through CloudConnector and just check if there is an active add-on purchase for Duo Enterprise for the namespace. This could be done with a single line of code:

GitlabSubscriptions::AddOnPurchase.by_namespace(...).for_duo_enterprise.exists? 

Regarding the cut-off date, I would totally go along with the suggestion from our colleague Nikola here.

changed the description

hi @hsnir1 - @vburton and I have put together the testing timelines for our next phase of Duo Enterprise testing. This phase of testing is specific to when we cut off our free access to AI features on 2024-10-17.

We will be separating this testing into 2 workstreams:

• The first is for testing SM at 17.4 from 2024-09-02 to 2024-09-09

Will this timeline work your team - AI Impact Dashboard? We will want to make sure this feature is included in the testing from 09-02 to 09-09. Does this timeline work for you and your team? Thanks! And let me know if there is a better place to ask this.

@cersoz provided that this issue has been resolved by then, the timeline should be okay.

@felipe_artur do you think we'll have this wrapped up in time?

cc @gweaver

@blabuschagne Yes, should be merged this week.

A status update, I am checking with fulfillment team if the solution of !163036 (merged) is valid.

Perfect, thanks @felipe_artur

@blabuschagne !163036 (merged) is merged. We will probably need to follow up with some permissions adjustments as discussed at #481525 (closed), but I think we are good for internal testing next week.

Amazing, thanks @felipe_artur

mentioned in issue gitlab-org/quality/triage-reports#19368 (closed)

mentioned in issue #480752 (closed)

added workflowin review label and removed workflowin dev label

changed the description

marked this issue as related to #481525 (closed)

added gitlab-org#15030 as parent epic

added workflowverification label and removed workflowin review label

@blabuschagne Looks like you already tested this. Can we consider this one verified?

@felipe_artur yeah I think so! Great work here

closed

added workflowcomplete label and removed workflowverification label

mentioned in issue #485506 (closed)

mentioned in issue #498497 (closed)

added devopsmonitor label and removed devopsplan label