Collect more usage data

@regisF try boiling this issue down to deliverables:

add stats data to EE
- add to the usage ping
- make sure this is reflected in the admin screen where you can opt-out (where we show what the current ping contains)
update version.gitlab.com
update terms of use
update privacy policy

They will almost all have different people working on them, so you might even want to make separate issues. Things like the terms of use and privacy policy can already be worked on.

I think most of 'general thoughts' should be moved to the description.

mentioned in issue gitlab-com/www-gitlab-com#800

mentioned in issue gitlab-com/www-gitlab-com#801 (closed)

Thanks @JobV for your comment. I've updated the body and created several issues.

@JobV @regisF I would like us to learn what versions and architectures of the runner our EE customers are using, to inform decisions about development efforts there.

It might look like Ci::Runner.online.group(:revision, :platform, :architecture).count

What do you think?

The burning question I have at the moment is: "how common are Windows runners among our customers?"

@nick.thomas

This is the current JSON the Ping feature sends.

{
   "version":"8.11.3-ee",
   "active_user_count":12,
   "license_md5":"X",
   "historical_max_users":13,
   "licensee":{
      "Name":"job.gitlap.com",
      "Company":"jobs",
      "Email":"job@gitlab.com"
   },
   "license_user_count":20000,
   "license_starts_at":"2015-05-15",
   "license_expires_at":"2050-01-01",
   "license_add_ons":{

   },
   "recorded_at":"2016-09-15T09:13:30.021-04:00"
}

How would you enhance it to include what you want to gather?

@regisF actually, how about: Ci::Runner.online.group(:version, :platform, :architecture).count? (version instead of revision).

The composite key is a little annoying, but we could put it into JSON like this:

"runner_permutations": [
 {"version":"1.6.0~beta.199.g6d8f982","platform":"windows","architecture":"i386","count":1},
 {"version":"1.6.0~beta.199.g6d8f982","platform":"linux","architecture":"i386","count":1},
 {"version":"1.6.0~beta.199.g6d8f982","platform":"linux","architecture":"amd64","count":5},
]

Total number of runners for this one customer would be like runner_permutations.map(&:count).sum

Total number of Windows runners for all our customers would be like:

most_recent_reports_per_customer
  .flat_map(&:runner_permutations)
  .select {|rp| rp.platform == "windows" }.map(&:count)
  .sum

Added ~91796 label

@nick.thomas a composite key is a bit complex, can you create a new issue for this and link it from https://gitlab.com/gitlab-org/gitlab-ee/issues/892 ?

We're going to try and ship this in 8.12 and we need to keep it as simple as possible.

We can ship it in 8.12 and ignore the extra data until we've improved version.gitlab.com The important thing is to ship the client side stuff in the release. That is the critical path because it takes months for customers to upgrade.

Mentioned in commit a7e4525e

Mentioned in merge request !735 (merged)

Mentioned in issue #1024 (moved)

Mentioned in issue gitlab-com/www-gitlab-com#801 (closed)

@regisF In !735 (merged), in addition to the above, we added:

Issue Boards
CI Runners
Deployments
Geo Nodes
LDAP Groups
LDAP Users
LFS objects
Protected branches
Releases
Remote mirrors

Mentioned in commit 55ce239b

Marked the task update terms of use gitlab-com/www-gitlab-com#801 (closed) as completed

@stanhu thanks, but when I look at the code, I see 30 data that are being collected and with your list above, we are at 27. 3 data are missing :-)

I think what we miss in your list is:

Webhooks
Services
LDAP keys

Can you confirm?

I'll also update the body issue about these new data.

Mentioned in commit 98538f9b

We don't have docs for this, I'll craft something.

@axil I've updated the list of data we gather in the body issue to reflect the latest changes.

Mentioned in commit 2db914a4

Mentioned in commit 5067244c

Status changed to closed by merge request !735 (merged)

@axil do we have the docs ready?

Mentioned in commit marcia/gitlab-ee@63260200

Mentioned in commit 941f7598

Milestone changed to %8.12

Marked the task add stats data to EE as completed

Marked the task add to the usage ping as completed

Marked the task make sure this is reflected in the admin screen where you can opt-out (where we show what the current ping contains) as completed

Marked the task update privacy policy gitlab-com/www-gitlab-com#800 as completed

Marked the task Update EE documentation about the ping feature as completed

Removed ~126459 label

added Enterprise Edition label

mentioned in issue #15944

Collect more usage data

Description

Specification

Designs

Child items ...

Activity