Run overlapping tests with confidence with Kameleoon's Cross-Campaign Analysis

In the world of experimentation, handling multiple simultaneous experiments is a common practice, but it has its unique challenges. Many practitioners advocate for isolating experiments to maintain integrity, but this approach can hinder a company's pace of innovation.

The consensus of experts like Ronny Kohavi, Georgi Georgiev, and Lukas Vermeer is that you should absolutely run multiple tests at the same time (and in the same zone), as the value of the insights far outweighs the unintended consequences.

To empower teams to run simultaneous experiments with confidence, we released Kameleoon's Cross-Campaign Analysis. This new feature illustrates the impact concurrent experiments have on each other.

In Kameleoon's reporting page, users can easily break down results based on:

• Exposure to other tests
• Identified conflicts
• Unintended effects

Read on to learn more about this powerful new feature, de-risk your overlapping experiments, and increase your test velocity

The Different Ways Managing Multiple Experiments

There are several common strategies for managing multiple experiments, each with its own tradeoffs:

1. Sequential testing: Experiments are conducted one after another, utilizing the entire user base. While this method ensures accuracy, it can be time-consuming and may require delaying subsequent experiments.
2. Mutually exclusive testing: Users are divided into segments and each segment participates in only one experiment. This approach maintains accuracy but reduces the power of experimentation. An important point to note here is that exclusive tests may still end up interacting, leading to inaccurate results. 
3. Overlapping or A/B/n testing: Multiple experiments are launched simultaneously, with the control group reused. This balances experimentation power and accuracy but requires launching and concluding experiments at the same time.
4. Multivariate testing: Similar to overlapping testing, this method compares all combinations of test and control against each other. It maximizes experimental power and accuracy but increases analysis complexity, making it less suitable for numerous tests.