▶ Full Post Text
Some background on me… I spent about 17 years in quant, mostly as a researcher / quant dev. My academic background is computer science, and at some point I picked up a CFA because when I first started I didn’t know anything about finance. Most of my career was institutional stuff… long horizons, low turnover, low tracking error portfolios. More enhanced indexing than pure alpha.
Now that I’m out and no longer need pre-clearance from compliance to trade stocks, I started looking at what retail traders are doing on the systematic side. I kept running into things like SMC and ICT. To me it felt like technical analysis with fancier names. That said, some people here do seem to make money with it, so I wanted to see whether there’s any real signal there or if it’s mostly data mining.
So I built a backtesting platform around `backtesting.py`. To get breadth quickly, I used an LLM to help translate a lot of these qualitative SMC/ICT “rules” into Python. It generated \~80 strategy variants… liquidity sweeps, FVGs, order blocks, ORB, Fibonacci retracements, etc. To be honest, I don’t fully understand half of them and I’m skeptical of most of it, but the goal was to test, not believe.
Once I had the strategies, I pulled an API I found on Reddit that tracks the most mentioned stocks across subs like r/wallstreetbets, r/stocks, r/investing, etc. I took the top 50 mentioned names and run all strategies across four timeframes: 5m, 15m, 1h, and 4h.
I have 1m OHLC data, but I skipped it for now. Feels like alpha probably decays too fast there, and I haven’t thought seriously about retail execution yet.
Single-name backtests run insanely fast compared to the portfolio optimization work I used to do in institutional quant (Axioma optmizer, Barra risk models, ITG transaction cost curves, etc).
Net result:
50 stocks × 80 strategies × 4 timeframes = \~16,000 backtests per run.
Lookback varies by timeframe:
* 5m → 14 days
* 15m → 30 days
* 1h → 60 days
* 4h → 180 days
I score each backtest using a composite that includes Sharpe, alpha return (vs buy & hold), win rate, number of trades (penalize higher turnover to loosely proxy costs), and max drawdown.
Obviously, if you run 16k backtests, you’re going to find some god-tier equity curves. My instinct is that I’m staring straight at a multiple-testing bias problem.
So a few questions for the group:
1. **Regime momentum:** My working theory is that these strategies aren’t evergreen, but might work during short-lived regimes (2 weeks on 5m, longer on higher timeframes). Has anyone here had success ranking strategies by recent performance and essentially riding the hot hand?
2. **Penalizing 16k trials:** I know Lopez de Prado talks about effectively deflating Sharpe by the number of tests run. I’ve been looking at the Deflated Sharpe Ratio, but I’m not sure if that’s overkill for heuristic-based retail strategies like this.
3. **OOS validity:** Is a 14-day lookback on a 5m strategy even long enough to justify any meaningful OOS test, or am I just looking at noise no matter what?
At this point I’m trying to figure out whether I’ve built a legitimate discovery engine… or if I’m just quantifying retail delusions with better tooling. Would love to hear from anyone who’s tried to bridge institutional risk discipline with faster-moving retail-style strategies.