FOIL
Counterfactual sensitivity profiling / Phase 0 / claude-sonnet-5

Measured 2026-08-13 · 200 samples per condition

The same four sentences.
One adjacent swap.
The answer inverts.

Ordering A
    1.000
    P(correct route)
    Ordering B
      0.005
      P(correct route)

      Both columns contain an identical set of scout reports — verified character for character. Only the sequence differs, and the two reports that changed places are both honest. The listener is near-deterministic inside each ordering: this is not sampling noise.

      What this is

      FOIL measures how much a decision actually depended on each information source, by forking the decision point, ablating one source, and re-sampling. Before building any of that, it runs three null experiments to find the harness noise floor, with a pre-registered rule that kills the project if the floor approaches the size of the signal.

      The rule fired at 36× the threshold. Reordering the inputs moved the measured decision further than removing an entire source did. The page below is the evidence, and it is the whole reason to be careful with ablation studies on language-model agents.

      Tnull — 95th pct across reorder + paraphrase0.995
      Tablate — median across single-source removal0.055
      Kill threshold — 0.5 × Tablate0.028
      Pre-registered verdictKILL

      Reorder the reports

      Drag a report, or use the arrows. The readout shows the measured action distribution for that exact ordering. Six of the twenty-four orderings were measured; the rest are blank, because this is data rather than a model of the data.

        Measured n = 200

        All twenty-four orderings

        Every permutation of the four scouts. Filled cells were measured; their fill is the basin the listener fell into. Outlined cells were never run. Select any filled cell to load it above.

        lands on the correct route lands on the unruled-out route not measured

        Why an ablation study should care

        Removing a source from a prompt changes two things at once: the information available, and the arrangement of everything after it. If arrangement alone can move the outcome further than removal does, an attribution score computed across differently-arranged coalitions is partly measuring the arrangement.

        The effect here is not a gentle drift. Five of six orderings agree; one inverts. A central summary — a mean, a median — hides that entirely, which is why this project now counts inverted orderings as a first-class statistic rather than averaging over them.

        Removing the liar
        0.010
        Removing an honest corroborator
        0.830

        Total-variation shift when each source is removed. The listener had already discounted the liar, so removing it changed almost nothing — while removing one of two byte-identical honest reports changed almost everything. Redundant sources are not interchangeable.

        What this does and does not show

        Established

        • Presentation order alone flipped a scored decision from 99% right to 99.5% wrong, on this episode and this model.
        • Within a fixed ordering, sampling variance is near zero. The bimodality is entirely between orderings.
        • Evaluating coalitions in a rearranged order — the project's main cost optimisation — is not viable.
        • Two sources emitting identical text are not interchangeable to the model.

        Not established

        • That ablation-based attribution is fragile in general. One episode, one seed, one model, one overlap setting.
        • That the residual metric fails. It was never computed — Phase 0 tests the harness, not the instrument.
        • That the paraphrase arm is clean. Its transform produces degraded English, confounding meaning with fluency. The ordering result carries no such confound.

        Status: Phase 0 not passed, under two protocols. The amended protocol — twelve episodes, a repaired percentile, and every route covered by at least one scout — returned INDETERMINATE. It removed the bistability driver and made the task over-determined instead: 81% of single-source ablations moved the distribution by exactly zero, leaving no signal for a noise floor to be measured against. Two environment failures in opposite directions; the instrument itself has still never been computed. The project does not advance to Phase 1.

        What the amended protocol found

        Twelve fresh episodes, every route covered. Order sensitivity did not go away — it concentrated. Episodes are not slightly order-sensitive; they are either immune or they invert completely.

        Episodes perfectly stable across all six orderings9 / 12
        Episodes that invert completely3 / 12
        Single-source ablations that moved nothing81%
        VerdictINDETERMINATE

        An unplanned observation

        The three episodes that invert under reordering are exactly the three where the listener’s answer is wrong — at ~1.00 confidence in every case. Nine stable episodes, nine correct.

        wrong
        right
        inverts under reorder
        3
        0
        stable
        0
        9

        If it holds, that is an error signal needing no ground truth, no normative baseline and no attribution: permute the inputs, and distrust an answer that moves. It would sidestep every limitation that made the original metric hard to defend.

        Treat this as a lead, not a result. It was noticed after the data came in, and the test was chosen after the pattern was visible, so the p = 0.0045 is descriptive rather than confirmatory. Twelve episodes with three positives is fragile. Ambiguity plausibly causes both the flipping and the error, which makes this a symptom rather than a mechanism — usable as a detector, unexplained as a cause. Establishing it needs a fresh pre-registration and a model and task family it was not discovered on.

        Run it yourself

        Python 3.11 and numpy. No other dependencies, deliberately — the harness has to run unchanged inside a lab, and every dependency is a reason someone doesn't. 

        # project the cost and inspect the exact request; zero API calls
python3 cli.py plan --show-prompt

# protocol v1: the run this page reports on (~2,200 calls)
python3 cli.py --n 200 run

# protocol v2: pooled across twelve episodes
python3 cli.py --n 50 run2 --episodes 12

        Exit code 3 means the kill rule fired. Determinism is at the request layer only: the environment, payloads and fork keys reproduce exactly, but responses are sampled and will not. The reported effect is large enough that it does not depend on resampling luck.