Add converted (boolean) column to the experiment_users table

For the most basic form of A/B testing we really need to record whether or not the user performed the desired action (i.e. converted).

Let’s add a boolean column to the experiment_users table which defaults to false and gets updated to true when the user converts.

We will also need to create some simple helper methods for updating the row for an existing user_id & experiment_id combination.

How it works

Here’s a rough idea of how adding such a column will give us much better insights using the “group-only trials” experiment as an example:

1. User arrives on the “select your namespace” step of the “start a trial” flow (i.e. /-/trails/select)

   • We know who this user is (i.e. we have a user record and can store their unique & constant user.id value) because they had to sign in or register to get this far

   • We immediately record them as being part of the experiment; that is to say, we insert a new row into the experiment_users table:

     id	experiment_id	user_id	group_type	converted
     38	4	10872	1 (experimental)	FALSE

     • Notice that we mark them as not having converted (because they really haven’t done anything yet, they’re just part of the experiment so far)

2. At any point in the future (it doesn’t have to be during this request or session, it doesn’t have to be in the same browser or on the same device) the user follows through with picking a namespace and starting their trial (by clicking on the “Start your free trial” button with a valid namespace selected)

   • As soon as that happens, we update their same row in the experiment_users table to specify that they have converted

     id	experiment_id	user_id	group_type	converted
     38	4	10872	1 (experimental)	TRUE

3. Once the user has converted for a specific experiment, we stop accepting updates for them.

---

This idea was originally mentioned here: #229057 (comment 380221114)