BL-D-004: The Evaluation Standard
BIJECTIVE LABS
BL-D-004: The Evaluation Standard
Status: RELEASED
Classification: PUBLIC
Revision: 1
Date: 2026-09-07
1. Scope
This article defines what has to be true before a measurement becomes a verdict.
It applies to every hypothesis in the record without exception, and it is the
standard the release gate in BL-FM-001 section 10 checks against.
It was written in the middle of the first programme, not at the start. Every requirement in it replaced a specific way the programme had already fooled itself.
2. The eight requirements
A measurement is not a verdict until all eight hold.
2.1 Net from day one. Every figure is computed after fees, spread, slippage,
and funding accrual where the position spans a settlement. Gross figures may be
reported alongside, labelled as gross, and may never carry the verdict. The cost
identity is in BL-D-011 section 4.
2.2 Correlation-aware inference. Returns are collapsed to a per-timestamp portfolio before any interval is computed, and intervals come from a moving-block bootstrap with a block length exceeding the holding horizon. Per-leg t-statistics are not permitted.
2.3 Sizing that matches the live path. The verdict is computed through the same sizing and risk logic the strategy would actually run under, not through equal weights. Equal-weight aggregation and capped-notional aggregation disagree by enough to flip a sign: one result in this archive reads +0.86 under raw equal-weight aggregation and -8.74 under the collapsed, sized construction [E-0022].
2.4 Subperiod stability. The sample is cut into at least three sequential
blocks and the effect must survive in each. A result that lives in one block is
reported as a Condition death, not as an edge.
2.5 Declared power. The sample size required to detect the pre-registered
effect is stated before the test runs. If the achieved sample falls short, the
outcome is NOT PROVEN regardless of which side of zero the point estimate
lands on.
2.6 A matched null. The result must beat a control that shares timing,
turnover, cost, and exposure with the strategy and differs only in the claimed
association. Construction is in BL-D-011 section 6.
2.7 Multiple-testing correction. Families of tests carry Benjamini-Hochberg correction. Sharpe ratios are deflated for the number of configurations tried. A naked Sharpe ratio is not reportable.
2.8 No selection after the fact. The universe, the era, the horizon, the rebalance phase, and the parameter grid are fixed at pre-registration. Choosing any of them after seeing outcomes is a look-ahead violation even when the signal itself is causal.
3. What each requirement is protecting against
The standard reads like boilerplate until each line is attached to the thing it caught.
| Requirement | The failure it exists to prevent |
|---|---|
| 2.1 Net from day one | An entire programme of strategies measured mid-to-mid, with a corrected expectation of -5.6 bps |
| 2.2 Correlation-aware | Fifteen instruments carrying the information of 1.33, treated as fifteen |
| 2.3 Live-path sizing | A sign flip between aggregation conventions |
| 2.4 Subperiod stability | A momentum result of +338 bps at p=0.001 that lived entirely in the final third |
| 2.5 Declared power | Underpowered results recorded as kills, closing branches nobody had tested |
| 2.6 Matched null | A time-shuffle control that manufactured 2,200 bps of apparent edge from nothing |
| 2.7 Correction | A best-of-N result reported as a single finding |
| 2.8 No post-hoc selection | A headline of +14.11 bps that was the maximum of 24 unreported rebalance phases |
The last row is the most recent and the most instructive, because the selection was not deliberate. Nobody chose the best hour. The hour was fixed early for an unrelated reason and never questioned, which produced the same statistical effect as choosing it. Requirement 2.8 exists to catch selection that happens by default rather than by intent.
4. The five-angle audit
Before any verdict is issued it passes an adversarial review across five angles. The review is run by default rather than on request, which is itself a rule adopted after the research team noticed adversarial review had only ever happened when someone specifically asked for it.
| Angle | The question |
|---|---|
| Calculation | Does the arithmetic reproduce from the artifact? |
| Cost | Is every cost term populated from a live source, and is any of them zero when it should not be? |
| Data | Does the instrument actually resolve the premise, at the required resolution and coverage? |
| Sample | Is the result concentrated in a symbol, a period, or a regime? |
| Mechanism | Does the economic story still hold given what was measured? |
The audit has a documented record of changing outcomes. It caught a volatility figure used to close two economic doors that was too small by a factor of 2.6. It moved a risk-gate lift from a reported 1.52x to 1.49x once the labels were made causal. And it produced two retractions of verdicts that had already been issued.
5. Verdict vocabulary
The audit produces one of five outcomes, and the vocabulary is fixed because free-form verdict language destroys comparability across thirty dossiers.
| Verdict | Conditions |
|---|---|
FALSIFIED | A pre-registered kill criterion met, with declared power achieved |
CLOSED | Effect measured real; a declared envelope parameter blocks it |
NOT PROVEN | Interval spans the threshold, or declared power not achieved |
OPEN | Under test, or awaiting a measurement not yet taken |
WITHDRAWN | A previously issued verdict, retracted |
FALSIFIED and NOT PROVEN are the pair that matters. The programme conflated
them once and the cost was a branch closed on statistics that had never been
computed on the conditioning partition the hypothesis itself named. The closure
was withdrawn and the branch was re-tested from four independent directions
before being closed again.
6. What the standard does not require
It does not require a result to be positive to be publishable. Twenty-seven of the thirty-seven settled verdicts in this record are falsifications, and they went through exactly the same audit as anything else.
It does not require certainty. NOT PROVEN is a legitimate terminal state for a
hypothesis, and five entries in this record hold it.
It does not require that a verdict be permanent. Every dossier carries reopening conditions in section 10, and an envelope-bound verdict states the parameter value at which it reverses. A standard that produced permanent verdicts from conditional measurements would be claiming more than the measurements support.
Copyright (c) 2026 Bijective Labs. All rights reserved.