Tests are running. A dashboard exists. Results get shared in a monthly email. Someone presents a win at the quarterly review. If you glance at it, you would conclude the organisation is experimenting and learning.
Then a new CFO arrives, or a cost review lands, or a new product leader asks a simple question: “What has this changed?” And the programme cannot answer. Within two budget cycles, it is gone.
You hired capable people. You invested in testing tools. You wrote a playbook. You ran workshops. You created Slack channels, hosted demo days, built onboarding materials, and maybe even flew everyone to the same city for a two-day alignment session.
And then you waited for it to work. But after some time, you notice a drift.
Every experimentation programme has the same quiet problem. The best ideas for what to test next do not appear when someone opens a planning tool and stares at a blank hypothesis field. They appear in the middle of a product review when someone says “we should test that.” They appear in sprint planning when a […]
Vanity metrics create a dangerous illusion of progress while masking the fundamental question: Is your experimentation program actually driving better business decisions, or is it just generating activity?
A year ago, we changed our name from Effective Experiments to Efestra. At the time, I wrote that it was not just a name change. It was the beginning of a fundamental rethink of what we were building, who we were building it for, and how we believed experimentation governance should work.
Today, that rethink is complete. This post is about what changed, what we held onto, and what Efestra looks like now.
Somewhere in your business, a team is reporting that experimentation generated several million pounds of revenue impact last year. It may sit under marketing, product, growth, digital or e-commerce, depending on how your organisation is structured. Wherever it sits, that number is being used to justify next year’s testing budget, the headcount on the team, […]
The experimentation industry advocates democratisation as the path to maturity. But giving more teams access to testing tools without decision intelligence infrastructure does not create distributed learning. It creates distributed chaos.
Most organisations run their experimentation programmes on hope.
Hope that teams will follow the playbook. Hope that insights will be shared. Hope that someone will connect experiments to strategy.
A company invests in testing tools, hires specialists, maybe even writes a playbook. Then they hand it all over and wait.
How practitioner-level thinking turns strategic governance challenges into tool selection exercises and how this impacts the organisations that hire them
Most organisations have built is elaborate experimentation theatre – expensive, impressive-looking, and fundamentally ineffective at driving the business outcomes you’re accountable for. The problem isn’t incompetence; it’s that the entire industry has conditioned us to measure maturity using metrics that sound sophisticated but predict nothing about business impact.
