Reporting Reinvented: Amplifying Experiment Outcomes with Deeper Analysis

“So...what exactly moved the needle?”

Everyone who’s ever run a multivariate test has likely been asked this question by senior management and struggled to give a firm answer. That’s because multivariate testing (MVT), by design, introduces so many variants of each element being tested that it’s nearly impossible to discern which variants actually made a difference.

The result? Many organizations, frustrated by the lack of visibility MVT provides, rarely test more than one element at a time and instead rely on slow, inefficient A/B testing.

How Better Reporting Makes a Difference

At Evolv, we’re focused on helping companies achieve exceptional revenue gains by dramatically accelerating their experimentation programs — and we know that in order for this to happen, companies need to be able to determine what moved the needle.

That’s why our platform includes intuitive data reporting dashboards to provide greater insights into your experiment results.

We designed these reporting views to help customers achieve three things.

First, we want to ensure customers have all the data they need to generate more ideas for new experiments. As Sam Nazari, our VP of Customer Success, said on our most recent “Think CX” webinar: “Optimization needs to happen in iterations. Take the data you get from one experiment and build on top of that to design the next round of tests.”

Second, we want to help customers get a clearer understanding of their experimentation results so they can communicate their findings to other members of the team. The intention is to further a culture of experimentation by involving more stakeholders in the process.

Finally, we hope these reports help customers receive value from every experiment. Not every test you run is going to have a single clear winner, but at least with more granular detail about what worked (or didn’t) and why, you’ll be able to see and understand the results so you can gain greater insight and come to intelligent conclusions.

How it Works

The power of Evolv reporting comes from Evolv’s ability to uncover variant-level insights. With variant-level insights, you’re able to go a step further than traditional MVT tools: instead of only being able to see the difference in performance between combinations of variants, you’re actually able to understand how each variant, on an individual level, affected the final outcome of an experiment.

As experiments run, Evolv constantly analyzes the results to extract the contribution of each individual change. Evolv’s AI algorithms can find and highlight the best changes, without you having to do guesswork or tedious data analysis.

These reporting views provide a ton of useful data for each variant. Selecting a value from the page or mobile app you’re optimizing will highlight the combination it was present in. You can then determine the most successful variants within a combination and use this information to determine what to test next. Additionally, for each variant, Evolv will tell you how many visitors have seen it, the probability that new visitors will see it, and the expected performance range of the variant at a 95% confidence level.

Learn More

These powerful data reporting features are available in the brand new Evolv Manager as part of our latest release of the Evolv platform.