Backtesting without proper WFA is mostly just curve fitting.
u/Kindly_Preference_54 ·
Reddit — r/algotrading
· March 03, 2026 at 18:07
· ⬆ 16 pts
· 💬 9 comments
| View on Reddit ↗
AI Summary
Summary
The post critiques the common backtesting approach in retail algorithmic trading, arguing it's often just curve fitting without rigorous validation.
The author's thesis is that a proper Rolling Walk-Forward Analysis (WFA) is necessary to validate a strategy's robustness and reduce the probability of it being curve-fit to historical data.
Quality assessment: This is a methodological discussion, not investment due diligence (DD). It provides sound, widely-accepted principles for quantitative strategy development but does not analyze any specific market or asset. It is high-quality advice for system developers, but contains no market speculation or noise.
Score16
Comments9
Upvote %84%
▶ Full Post Text
I see many posts saying:
“I backtested several years. It works. Now I’ll go paper. If paper works, I go live.”
But when people say “backtested”, they usually mean they tried different parameters several times and chose the best settings. That’s actually limited manual optimization. The problem is they don’t know if the result is just curve fitting. This needs to be refuted.
Most likely outcomes:
* It fails already on paper -> wasted time
* It survives paper by luck -> fails live -> real money lost.
So how do you reduce the probability it’s curve fit? Rolling Walk-Forward Analysis (WFA).
Example (simplified):
1. Sep 2024 – Feb 2025 (in-sample - IS): full optimization + define selection criteria (PF, Sharpe, Recovery Factor, etc. + backward OOS can also serve as criterion).
2. Mar – May 2025 (out-of-sample - OOS): test the selected setup. If fails, change selection criteria.
That’s one WFA round.
Now repeat this process across past data. Not once - many times. Most traders effectively perform one WFA round with the OOS being “the future”. But you can perform many WFA rounds historically and build a statistically meaningful sample. If a strategy survives 12 WFA rounds, what are the chances it won’t survive the 13th?