Evidence standards and human decisions

Outcome: establish practical standards for when a team may act on a growth claim. More reports do not imply better evidence. A useful system distinguishes trustworthy observations, promising hypotheses and incomplete information.

Maintain three levels of evidence

Verified for purpose means the relevant measure, time range, source and reconciliation checks meet the decision's needs. Provisional means the observation is credible but could change with a known limitation, such as missing refund costs or consent-biased tracking. Blocked means the evidence cannot safely support the decision, perhaps because the source is stale or permissions have been lost.

These are operational judgments, not universal scientific grades. A minor email-copy test may justify different checks from a storewide pricing change. Make the required standard proportional to the cost of being wrong.

Keep AI in the draft lane

AI can summarize source-linked evidence, suggest hypotheses, draft priorities and identify contradictions. It should not silently change accepted metrics, approve a commercial strategy or authorize a live operation. Preserve the original source and distinguish the model's suggestion from the human operator's decision.

When a team changes its mind, capture the trigger and reason rather than silently rewriting history. The distinction between an accepted insight and a proposed action makes audits, coaching and learning possible.

Worked decision: inconsistent sales totals

Suppose Shopify reports a different amount of revenue than a marketing analytics tool. The difference may reflect cancellations, tax, currency conversion, session attribution or missing consented events. If the question is financial profitability, use the order and accounting source appropriate to that decision and reconcile relevant adjustments. If the question is attributed acquisition, document the attribution limitations rather than pretending the two numbers must be identical.

Write an evidence contract

  • Name the decision, metric definitions and system of record

  • Set freshness, completeness and reconciliation checks for the risk level

  • Flag missing sources, uncertainty and known biases in visible language

  • Require a named approver for accepted priorities and releases

  • Preserve reasons for overrides and reopen decisions on material new evidence

Self-check: could a skeptical colleague identify which claims are supported, which are provisional and which cannot yet justify action? If yes, your process is less likely to confuse speed with competence.

Explore the reference

01 · The four-step Growth System

02 · Nine connected growth levers

03 · Evidence standards and human decisions

04 · Operating cadence and continuous learning