We tried to break our own diagnostic

An instrument you have never attacked is not an instrument, it is an assumption. So we spent a week building the strongest attack we could against our own martingale-collapse detector. It held. Then almost everything else we believed came apart.

The detector exists because prediction-market prices are close to a martingale, so a model can drive its objective down by learning something very near the identity map. That map has full rank and full per-dimension variance, so it passes every standard collapse check while encoding nothing. Our copy oracle scores the model against an explicit copy of the nearest visible embedding, and on real Kalshi strike ladders it reported alignment of 0.964. Copying, everywhere.

Then we found a reason to doubt our own number. The target encoder ran unmasked attention across the whole window, so under temporal masking the target at a visible patch was computed with attention over the masked ones. The oracle's reference was contaminated with the future. If that was what made copying look easy, the detector was measuring our architecture rather than the market, and a month of conclusions would have to go. The test is clean: make the time axis of the target tower causal, change nothing else, run the two-by-two over which towers are causal, four seeds per cell.

The contamination was real. Reading identical weights out both ways, the two target towers genuinely differ, by 12.7 percent in relative L2 at the earliest patch. And removing it changed nothing that mattered: copy alignment moved from 0.9641 to 0.9616, probe R² from +0.3919 to +0.3924, a Welch t of 0.09. We built the attack, confirmed the flaw it was aimed at, and the detector did not move. The copying is the market.

So we went looking for what else we had never checked. Every conclusion rested on a six-hundred-step budget nobody had pushed past, so we pushed: three seeds, 19,200 steps, probed at six checkpoints against an untrained encoder of the same architecture.

Thirty-two times the compute bought nothing. The ridge probe is flat, +0.4006 to +0.3965. The nonlinear probe peaks at the very first checkpoint and falls 0.091 from there, ending 23 pooled standard deviations below the untrained floor. The part we find hardest to unsee is the second line: effective rank nearly triples over the same run, 36 to 94, while utility falls. Rank is the standard check for representational collapse, and here it is not merely blind to the failure, it points confidently the wrong way.

Nor is the copy oracle any better as a guide. Both anti-collapse regularisers this programme had written off turned out never to have been switched on: each was computed on a detached target, in two separate repositories, contributing exactly zero gradient to every run either of them ever did. Put them back in and they work, and copy alignment falls from 0.965 to 0.776. Nothing improves; at the strongest settings the MLP probe gets worse. Across twenty-seven runs the diagnostic spans a range of 0.198 while probe R² spans 0.035, and the correlation between the two is positive. We can set the number almost anywhere we like with one hyperparameter, and nothing downstream follows.

That left the comparison the whole programme was built on, and it did not survive either. We had been ranking every model by a mean over four probe targets. One of them is documented in our own data code as a derivable function of the input, a sanity check whose low values mean the encoder is broken rather than that the result is interesting, and nearly the entire gap between our models and raw features lives there: the best raw arm scores +0.9407 on it against the best learned readout's +0.7552. A compressed embedding cannot beat the raw input at reproducing a function of that input, so it was never a comparison anyone could win.

On the one target that is genuinely about the future, every arm we have ever run lands between −0.004 and +0.051. Raw features, untrained encoders and every trained model, indistinguishable from each other and from zero. The number the programme was optimising was largely measuring how well a model reconstructs its own input, and the number anyone would actually trade on was flat the whole time.

None of which means the objective is broken. On a simulator where we control the hidden state and know it is recoverable, the same objective beats raw features comfortably, and beats them by more as the observations get noisier.

Across a thirty-two-fold range of quote noise the advantage never crosses zero and more than doubles, from +0.0690 to +0.1502. Raw features fall apart, +0.8239 to +0.6734, while the learned representation barely moves, +0.8928 to +0.8237. Where there is hidden state to recover, this class of model recovers it and keeps recovering it long after the raw cross-section has stopped being usable. Our target set simply had none in it.

We are stopping here rather than reaching for a bigger model, because the experiments that would have justified one came back negative. What would restart this is not more parameters or more steps, it is a target with something in it: realised volatility over a future window, or the settlement of a different correlated market.

None of this came from a new idea. It came from varying a readout we had never varied, matching control widths we had never matched, decomposing a mean we had quoted for a month, and checking whether a regulariser we had dismissed was ever running. Attack your instruments before you trust them. Then check what they predict, because an instrument can be perfectly sound and still measure nothing you care about.

Every number here is reproducible from physicalreasoning/causal-jepa, which carries the code, the raw results and the full findings record, including the corrections and the two pre-registered gates we got wrong.