A backtest is an argument, and like any argument it deserves auditing before it earns your trust. Four quiet assumptions appear in most of them — rarely from bad intent — and each one moves the result the same direction: up. Knowing them turns you from an audience into an auditor.
1. The fill assumption
The largest lie is usually a single line of configuration: where the simulated order fills. Filling at the touch of a limit price assumes you had queue priority you never had — in a real book, a limit at the low may not fill at the touch, because that depends on your place in the queue. The safe test assumes it fills only if price trades through it, and the fills you do get are biased toward the trades you least wanted. Filling market orders at the last price ignores the spread you pay every single time.
On an intraday strategy doing dozens of trades, a one-tick-per-side fantasy compounds into the entire edge. Plenty of published equity curves are nothing but the fill assumption, plotted.
2. The data assumption
A strategy tested on bar closes sees four prices per bar and infers everything between them. Whether the high or the low came first inside the bar is invisible — so any logic where a stop and a target could both have been hit in one bar is being resolved by the simulator's guess, and the guess systematically flatters. Order-flow logic is worse: without the actual sequence of trades, "delta" reconstructed from bars is an estimate wearing the costume of a measurement.
The honest versions of these tests run on recorded tick sequences. They are slower, the data is bigger, and the results are worse — which is precisely why they are rarer.
3. The selection assumption
Tune a strategy's parameters until the curve looks best, then present that curve, and you have not measured the strategy — you have measured your own persistence. With enough parameters, some combination profits on any historical window, including on random data. The curve is real; the inference is fake.
The tell is a result that only exists at one setting. A real effect degrades gracefully as you move the knobs; an artifact of the search collapses. If nobody shows you the neighbouring parameter values, assume they were worse — because if they weren't, they'd be in the picture.
4. The regime assumption
Every backtest is a bet that the future resembles the tested window. Some windows are generous: one long trending year can make any breakout system look inevitable. The minimum honest standard is performance across regimes — trend and chop, high and low volatility, including stretches where the strategy should struggle. A curve that never struggles wasn't tested; it was curated.
What this means for reading anyone's results
- Ask what the fill model was. If the answer is vague, the answer is "optimistic."
- Ask what data resolution it ran on. Bars for tick-level logic is a red flag by itself.
- Ask how many variants were tried before this one. Silence is a number, and it is large.
- Ask to see the bad stretch. Every honest record has one.
We hold our own simulations to the pessimistic side of every one of these lines — resting orders must be traded through, takers pay the spread, stops slip — because a simulated result that flatters you is not a feature. It is the most expensive kind of bug: one you'll fund with real money.