Be clear about multiple variant vs. multivariate

What does this MR do?

Clarifies that we support multiple variant (multi-variant) experiments rather than multivariate experiments.

What’s the difference?

Multi-variant experiments typically have variants beyond the standard A (control) & B (candidate). Each variant can be thought of as a completely different hallway through which the user can travel to get from point A to point B. For example, we could have a single button that currently has a green color and we want to test the effect of changing it to a blue color, a red color, and a yellow color.

Thus the user would receive one of the following four possible experiences:

• control: green button
• candidate 1: blue button
• candidate 2: red button
• candidate 3: yellow button

We would then simply need to record the name of the variant received and whether or not the user clicked the button (or whatever our success metric is).

Multivariate experiments are somewhat similar to multi-variant, but they actually contain multiple variables within each test. These tests are more like a single hallway with different options for wall color, floor type, and wall art. To further our previous example, we could have a button which is currently green, but we’ll also test different font sizes of the text on the button as well as what that text says. We’ll plan out a number of different options for each variable and then run a mesh of all of those variables in all possible combinations.

Let’s say we have the following variables and the different values we want to test:

• button color: green, blue, red
• text size: 11px, 12px, 14px
• text content: "Buy now", "Purchase", "Checkout"

Then we’d have the following matrix of possible combinations that would be tested:

Text size	Button color	Text content	Text size	Button color	Text content	Text size	Button color	Text content
11px	Green	Buy now	12px	Green	Buy now	14px	Green	Buy now
		Purchase			Purchase			Purchase
		Checkout			Checkout			Checkout
	Blue	Buy now		Blue	Buy now		Blue	Buy now
		Purchase			Purchase			Purchase
		Checkout			Checkout			Checkout
	Red	Buy now		Red	Buy now		Red	Buy now
		Purchase			Purchase			Purchase
		Checkout			Checkout			Checkout

Each participant of the experiment would receive one of those 27 possible combinations and we’d report on which values that person got for each variable as well as whatever outcome/success metric we’re using to validate the experiment (could simply be whether or not they click the button).

Does this MR meet the acceptance criteria?

Conformity

• 📋 Does this MR need a changelog?
  • I have included a changelog entry.
  • I have not included a changelog entry because its a very simple documentation update.
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• [-] Database guides
• [-] Separation of EE specific content

Availability and Testing

• [-] Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• [-] Tested in all supported browsers
• [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• [-] Label as security and @ mention @gitlab-com/gl-security/appsec
• [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• [-] Security reports checked/validated by a reviewer from the AppSec team