Backtesting Without Fooling Yourself — Low-Latency Trading Systems & Algo Trading — From the Order Book to a Live Retail Algo

If this course could give you only one skill, it would be this one: backtesting honestly. Backtesting — simulating a strategy on historical data to see how it would have done — is how you evaluate ideas without risking money. But it’s also where traders fool themselves most spectacularly, building beautiful equity curves that evaporate the instant real money is involved. The difference between a backtest that helps you and one that ruins you is whether you were honest — and honesty here is a set of specific, learnable disciplines. I ran real demos (on pure random data, so there’s no edge to find) to show exactly how the lies work.

Why Backtest At All

First, the upside, because backtesting done right is genuinely powerful:

Reject bad ideas cheaply. Most strategy ideas are bad. A backtest kills them in minutes for free, instead of over months with real money. This filtering is most of a backtest’s value.
Understand behaviour, not just returns. A backtest shows you the drawdowns (how much you’d have been underwater, and for how long), the win rate, the worst day, the distribution of outcomes. You need this to size positions (Lesson 12) and to know what “normal” looks like so you don’t panic live.
Build justified conviction. To follow a system through an inevitable losing streak (Lesson 12), you need to believe in it. A rigorous backtest — with an economic reason for the edge (Lesson 2) — is how you earn that conviction honestly, rather than abandoning a good system at the first drawdown.
Tune within reason. You can explore parameter choices — but carefully, because this is exactly where overfitting creeps in (below).

Backtesting is essential. The catch is that a dishonest backtest is worse than none, because it gives you false confidence to bet real money on nothing. So the rest of this lesson is the three ways it lies and how to stop them.

Lie #1: Lookahead Bias

Lookahead bias is using, at some point in your backtest, information you would not actually have had at that moment. It’s the most common and most devastating backtest error, because it’s subtle and it makes garbage look like gold. I demonstrated it on a pure random price series — data with no real edge to find at all.

The demo: the same simple strategy idea, on the same random data, run two ways. The version that (through a one-line indexing slip) used today’s return to decide today’s position — peeking at information it couldn’t have had — “returned” +1605%, a money printer. The honest version, using only yesterday’s data to decide today’s trade, returned −19% — the real, useless result you’d expect from trading noise. One tiny lookahead turned random noise into a fantasy fortune.

How lookahead sneaks in:

Using a bar’s close to decide a trade within that same bar. At the moment you’d trade, you wouldn’t yet know the bar’s closing price. Decide using only completed information.
Computing a statistic over the whole dataset, then “testing” inside it. Normalising prices using a mean/standard-deviation computed over the entire history (including the future) leaks future information into the past.
Survivorship bias (Lie #2, below) is a cousin — using knowledge of which instruments survived.

The fix is a strict rule: at bar t, your strategy may use only data up to t−1. Feed data forward through time, and never let the strategy see anything it couldn’t have known at that instant. A point-in-time data discipline (timestamp on arrival, Lesson 3) enforces this. And a heuristic that will save you repeatedly: if a backtest looks too good to be true, suspect lookahead first — it almost always is. Spectacular, smooth, high-return backtests are far more often bugs than discoveries.

Lie #2: Survivorship Bias

Survivorship bias is testing only on the instruments that still exist today. If you backtest a stock strategy on the current index constituents, you’ve silently excluded every company that went bankrupt, got delisted, or was removed — the losers. Your universe is pre-filtered to winners, so almost any strategy looks good. The fix is to use point-in-time universe data: test on the instruments that existed at each point in the past, including the ones that later failed. This bias is quieter than lookahead but systematically inflates results, especially for anything holding positions over longer horizons.

Lie #3: Overfitting

Overfitting is tuning your strategy so hard to the historical data that you fit its noise rather than any real signal — and noise doesn’t repeat. Again I measured it on data with no real edge.

The demo: I searched 30 different threshold values on one slice of data and picked the best — it returned +11.8% on that slice. Then I applied that same best parameter to unseen data: −12.4%. The “optimum” was an illusion of the one sample I tuned on; it captured that slice’s noise, which didn’t recur. The more parameters you tune and the more combinations you try, the worse this gets — you’re increasingly fitting randomness.

The defenses, which together keep a backtest honest:

Out-of-sample testing. Hold back a chunk of data you never look at while developing, and only test the finished strategy on it once. If it collapses out-of-sample, it was overfit.
Walk-forward testing. Repeatedly tune on a past window and test on the next unseen window, rolling forward — a more realistic simulation of how you’d actually adapt over time.
Fewer parameters, simpler rules. Every parameter is a chance to overfit. Prefer simple, robust rules with an economic reason (Lesson 2) over a pile of fitted magic numbers. A strategy you can explain is less likely to be a fit to noise.
Realistic costs and slippage. Apply honest transaction costs (Lesson 12) in every backtest — many “edges” are smaller than their costs and vanish once you’re realistic.
Suspicion of perfection. A gorgeous, smooth, high-Sharpe equity curve is a red flag, not a triumph. Real edges are noisy and modest.

The one-line summary worth taping to your monitor: a strategy that only works on the data you built it on is not a strategy — it’s a memory.

Pitfalls to Avoid

Lookahead bias. Using data from the future (even the current bar’s close). At bar t, use only data up to t−1. Suspect it whenever results look amazing.
Survivorship bias. Testing only on instruments that still exist. Use point-in-time universes including the ones that failed.
Overfitting. Tuning to noise. Few parameters, out-of-sample and walk-forward testing, an economic reason for the edge.
Zero or optimistic costs. Backtest with realistic brokerage, charges and slippage (Lesson 12), and honest fills (Lesson 4) — or your edge is fiction.
Trusting a perfect equity curve. Smooth and spectacular usually means buggy, not brilliant. Be suspicious.
No economic rationale. An edge you can’t explain (Lesson 2) is probably overfit. Demand a reason.
Tuning on your test set. The moment you optimise against your “out-of-sample” data, it’s no longer out-of-sample. Keep a truly untouched holdout.

Quick Recap

Backtesting is essential — it rejects bad ideas cheaply, reveals drawdowns, and builds justified conviction — but a dishonest backtest is worse than none.
Lookahead bias (using future info) is the deadliest: measured, it turned random noise into +1605% vs −19% honest. At bar t, use only data up to t−1.
Survivorship bias (testing only survivors) and overfitting (tuning to noise: +11.8% in-sample → −12.4% out-of-sample) both inflate results.
Defenses: point-in-time data, realistic costs/fills, out-of-sample + walk-forward testing, few parameters, an economic reason, and suspicion of perfect curves.
“A strategy that only works on the data you built it on is a memory, not a strategy.” Educational, not financial advice.

Next lesson: the strategies themselves — market making, momentum, mean reversion, stat arb — and a serious look at options selling, with a short-straddle example and its risks.