efestra
LoginRequest a call

Resources

The Evidence Gap explained

The Evidence Gap is the distance between the evidence an organisation produces, through experiments, research and analysis, and the decisions that evidence should inform.

What the Evidence Gap is

The Evidence Gap is not a single point of failure. It spans three things: whether the evidence can be trusted, whether it can be connected across teams and over time, and whether it reaches the people making decisions.

Most organisations assume the problem is producing better evidence, so they buy more sophisticated testing tools, hire more analysts and run more experiments. The gap rarely sits there. It sits in what happens after the evidence is produced.

The result is a widening distance between what organisations spend on evidence and the value they get from it. Teams run hundreds of experiments a year and produce a large volume of research, yet leaders cannot say with confidence which decisions that work shaped.

The Evidence Gap is about the missing structure that makes evidence trustworthy, connects it across the organisation and carries it into the decisions it should inform.

The three layers

The Decision Intelligence Stack describes the three layers needed to close the gap. Governance runs through all three.

  • Layer 1: Evidence integrity

    Can we trust what we have produced?

    Experiments are designed properly, do not collide with each other, use sound methods and are interpreted correctly. Research uses appropriate samples and its conclusions are supported by the data. If this layer fails, every decision built on it is compromised.

  • Layer 2: Evidence synthesis

    Can we connect what we know across teams, studies and time?

    Past evidence can be found, duplicate work is avoided and knowledge builds up over time. In most organisations experiments sit in testing tools, research sits in slide decks and customer data sits in analytics platforms, and none of it is connected. Few tools address this layer.

  • Layer 3: Decision influence

    Does trustworthy, connected evidence reach the decisions that matter?

    Decision rules are agreed before experiments launch, evidence is visible to decision-makers, dashboards show which decisions evidence informed, and someone is accountable for acting on it. Without this layer, even sound and well-connected evidence does not reach the boardroom.

  • Governance

    Governance is not a separate layer. At layer 1 it is quality assurance. At layer 2 it is knowledge standards. At layer 3 it is accountability for decisions.

Signs of the gap

  • Evidence integrity

    • Winning tests are not implemented as tested, or not implemented at all.
    • Leaders question whether experiment results are reliable and hesitate to act on them.
    • Teams use different statistical standards and definitions of success, so results cannot be compared.
    • Teams report test volume and win rates but cannot show business impact.

  • Evidence synthesis

    • Findings from six months ago cannot be found, so studies are repeated or decisions are made without them.
    • Teams repeat experiments or research already done elsewhere in the organisation.
    • Insights stay within the team that produced them.

  • Decision influence

    • Experiments and research are tied to tactical metrics, not to the decisions they should inform.
    • The programme cannot show what it contributed, which leaves it exposed when budgets are cut.
    • Practitioners report in statistical significance and conversion rates; executives think in revenue and priorities. The findings are not translated.

If your organisation runs experiments but cannot show the board which decisions they influenced, you have an Evidence Gap. The cause is usually missing structure, not the quality of your team or your tools.

Why it happens

  • Disconnected tools. Experiments are tracked in Airtable, implemented through Jira, documented in Notion and reported in spreadsheets. With no single record, insights are lost between tools and outcomes cannot be traced. This patchwork is sometimes called the Frankenstack.
  • Inconsistent methods. Without shared standards, teams set their own sample sizes, test durations and definitions of success, and nobody checks experiment design.
  • Work not linked to objectives. Tests are chosen for tactical metrics, with no clear connection to OKRs or to the decisions leaders face.
  • No tracking after release. Nobody checks whether winning variants were implemented as tested, or whether they delivered the expected value.
  • Each tool covers one piece. Testing platforms run experiments, research tools organise one type of evidence and BI tools display data. None of them connects evidence to decisions.

Questions to ask about your organisation

  1. What share of your winning experiments are implemented exactly as tested?
  2. Can you track the business impact of an experiment three to six months after release?
  3. How often do teams unknowingly repeat earlier experiments?
  4. Do executives refer to experiment results when making strategic decisions?
  5. How many different tools does it take to manage your experiments and research?

The self-assessment asks ten questions like these from a leader's position, and shows what an audit would examine for each one you cannot answer.

Take the self-assessment

Closing the gap

Flywheel

A platform that checks evidence as it is produced and links it to decisions.

About Flywheel