A/B Test

Drag and drop one creative group onto another to use split-testing functionality within the tree editor.

Cohorts or Bucketing Users for Testing

There are multiple ways of bucketing users for the purpose of testing or creative distribution across volume:

1. People Based - Hashed algorithm of the user ID. Use an integer between 0-100. Use a hyphen to separate the value. Example: 10% user cohort can be 0-10, 10-20, 20-30.

Multiple cohorts: To avoid any small gaps, repeat the end number in the next range.

Example:

  • Name: 10% PSA; 0-10
  • Name: 10% Control; 10-20
  • Name: 80% All Else; 20-100

2. Random - Thunder uses the random function (like rolling a dice) off the user ID to decide whether to bucket a user into a given impression quota.

As an example, if you want to create 4 buckets of 25% each, you have to create 1 bucket with 25%, then 1 bucket with 33% (⅓ of the remaining 75%), then 1 bucket with 50% (½ of the remaining 50%) and then 1 bucket with 100%.

3. Impression - Thunder uses the "random" function of the request ID rather than user ID to decide whether to bucket at user into a given impression quota. It’s like random, but more intuitive. Example: 50% of impressions go to creative group A.

Did this answer your question?