You hired capable people. You invested in testing tools. You wrote a playbook. You ran workshops. You created Slack channels, hosted demo days, built onboarding materials, and maybe even flew everyone to the same city for a two-day alignment session. And then you waited for it to work. But after some time, you notice a drift.
Vanity metrics create a dangerous illusion of progress while masking the fundamental question: Is your experimentation program actually driving better business decisions, or is it just generating activity?
A year ago, we changed our name from Effective Experiments to Efestra. At the time, I wrote that it was not just a name change. It was the beginning of a fundamental rethink of what we were building, who we were building it for, and how we believed experimentation governance should work. Today, that rethink is complete. This post is about what changed, what we held onto, and what Efestra looks like now.
The experimentation industry advocates democratisation as the path to maturity. But giving more teams access to testing tools without decision intelligence infrastructure does not create distributed learning. It creates distributed chaos.
Most organisations run their experimentation programmes on hope. Hope that teams will follow the playbook. Hope that insights will be shared. Hope that someone will connect experiments to strategy. A company invests in testing tools, hires specialists, maybe even writes a playbook. Then they hand it all over and wait.
How practitioner-level thinking turns strategic governance challenges into tool selection exercises and how this impacts the organisations that hire them
Most organisations have built is elaborate experimentation theatre - expensive, impressive-looking, and fundamentally ineffective at driving the business outcomes you're accountable for. The problem isn't incompetence; it's that the entire industry has conditioned us to measure maturity using metrics that sound sophisticated but predict nothing about business impact.
Learn how Decision Protocols prevent post-experiment paralysis by eliminating unreliable post-hoc decision making in experimentation programmes.
Recent posts
-
Why Workshops, Playbooks, and Good Intentions Will Never Scale Your Experimentation Programme -
Introducing Mosaic Companion: Because Your Best Experiment Ideas Happen in Meetings, Not in Backlogs -
Beyond Vanity Metrics: Measuring Experimentation Governance for Strategic Impact -
One Year From Rebrand to Rebuild: What Efestra Looks Like Now -
The CFO’s Blind Spot: Why Experimentation Revenue Numbers Don’t Survive Audit
