Curve fitting is tuning a strategy until it matches the past instead of capturing something that repeats. It needs no bad intent: try enough settings, and one of them will fit any stretch of history beautifully, by chance. The fitted strategy then trades the future the way a lucky coin flips the next time.
Why the best result flatters itself
Every combination a search tries is a lottery ticket. Most of a strategy's results are noise around its real edge, and the more tickets you draw, the further the luckiest one sits above that edge. So the best of 500 combinations looks better than the best of 5 even if the strategy is identical. The question is never "what was the best result?" but "how much better is it than the luck of trying this many?"
Habits that protect you
- Keep the logic simple. Every extra rule and setting is another way to fit the past. A strategy with two settings that matter is far more likely to hold than one with ten.
- Prefer plateaus to peaks. A setting whose neighbors do nearly as well is describing the market; one that works only at exactly 14.5 is describing the sample. Choose the middle of a broad good region.
- Search less than your trades allow. A few hundred trades can fairly compare a few dozen combinations, not thousands.
- Decide the yardstick first. Choosing what "best" means after seeing the results lets the search pick its own scoring.
- Keep data unseen. Test the finished strategy once on months you never built or tuned it on, and don't tune it again to pass.
- Test across conditions. A strategy that works only in one year's market is fit to that year.
The checks AlgoLift builds in
You don't have to remember all of that: the product holds you to it.
- The search budget compares a plan's size to its trades before it runs, and says Mostly luck when the winner would mostly be chance. See The Search Budget.
- The yardstick locks at the project's first search, so every later search ranks the same way.
- Results discount the winner for every combination tried: a sweep's verdict reads its deflated Sharpe, the chance its best result is overfit, and how broad its plateau is. Later versions of a search count every earlier try too.
- The planned split holds a sweep's last months back, so a walk-forward afterwards is graded on months the sweep never saw. See The Planned Split.
- A walk-forward grades every choice on months it never saw. See Walk-Forward.
- The holdout keeps the project's most recent months sealed for one final test, and the Scorecard caps any project that has touched them, or has no test on unseen data, below Robust.
- The Luck Test checks whether a strategy's entries beat random timing before it is tuned at all: a strategy with no edge has nothing for a search to find.