BL-IS-002: Permutation and Sign-Permutation Null

Status: RELEASED

Classification: PUBLIC

Revision: 1

Date: 2026-09-07

1. What it measures

Whether a result is better than the same strategy with its claimed edge removed and everything else left intact.

That phrasing carries the whole design. A null control is only informative if it differs from the strategy in exactly one respect. Break anything else and the comparison measures the wrong thing.

2. The failure that defined the construction

An early null in this programme shuffled the time index. Against that control, a strategy appeared to produce roughly 2,200 basis points of edge.

There was no edge. Shuffling time destroys the autocorrelation structure of the return series, so the shuffled series is a different object from the real one in ways that have nothing to do with the signal. The comparison was between a strategy on real data and a strategy on data that no longer resembled a market.

The lesson is narrower than "nulls are hard". It is that a null which changes more than one thing produces a number, and the number is meaningless in a way that is not visible from its magnitude.

3. The two constructions used

Sign permutation, for directional strategies.

eps_t  drawn independently from { -1, +1 }
s_t    ->  eps_t * s_t

The signal keeps its timing, its magnitude, its clustering, and its turnover. Only the association between signal and subsequent return is destroyed.

Cross-sectional shuffle, for panel strategies.

within each timestamp t, permute the weights w_{i,t} across instruments i

The portfolio keeps its gross exposure, its turnover, its rebalance schedule and its cost. Only the mapping from instrument to weight is randomised.

Both preserve everything the strategy does except the one thing it claims.

4. What is reported

Three numbers, together:

observed      the strategy result
null_mean     centre of the null distribution over B permutations
null_sd       dispersion of the null distribution
p_perm        share of permutations at or above the observed value
sigma_above   ( observed - null_mean ) / null_sd

The null mean is reported rather than assumed to be zero, and that matters. On a cost-bearing strategy the null distribution centres near minus the round trip, because a randomised strategy still pays to transact. A null centred at zero is a sign the cost model was not applied to the control.

Two examples from the record:

Both nulls sit well below zero, which is what a correctly costed control looks like.

5. The trap in those two rows

Both results beat their null by more than five standard deviations. Neither was accepted as an edge.

The first has a confidence interval of [-3.17, +6.49] and the second [-3.05, +12.67]. Both span zero.

Beating a null and having a positive expectation are different claims. The first says the strategy is doing something other than trading randomly. The second says that something is worth money. A strategy can clear the first bar decisively and fail the second, and these two rows are exactly that case.

Reporting the null distance without the interval would have made both look like findings. R-06 requires both, and BL-D-004 section 2 lists them as separate requirements for that reason.

6. Interface

permutation_null(strategy_fn, data, mode, n_perm, cost_model) -> NullReport
 
  strategy_fn   the specification under test
  mode          SIGN or CROSS_SECTIONAL
  n_perm        B, at least 1000
  cost_model    the same model applied to the strategy, per BL-IS-005
 
Raises when:
  the cost model passed to the null differs from the one used on the strategy
  n_perm < 1000
  the permutation changes turnover by more than a declared tolerance

The first check exists because applying costs to the strategy and not to the control is the most common way this instrument is used incorrectly, and it flatters the strategy by exactly the round trip.

The third is a guard on the construction itself. If a permutation materially changes turnover, it has changed something other than the signal, and the null is no longer matched.

7. What it does not do

It does not correct for multiple testing. A permutation p-value is a single test and inherits every multiplicity problem the programme has. R-05 handles that.

It does not establish a mechanism. Beating a matched null says the strategy is not random. It says nothing about why, and a result that beats a null while failing its own mirror test has an association without a cause. That is what happened to the positioning-drift hypothesis, which never reached this instrument because the mirror test closed it first.

It does not survive a substrate the strategy has already been fitted on. Permutation tests the association, not the search that found it.

8. Cross-references

BL-D-011 section 6 for the construction. BL-D-004 sections 2.6 and 2.5. BL-D-009 rules R-03 and R-06. BL-IS-001 for the interval reported alongside. Evidence E-0025, E-0026.

Copyright (c) 2026 Bijective Labs. All rights reserved.