Overfit Likelihood

Overfit Likelihood asks one question a backtest's own statistics can't always answer: was this strategy found because it works, or was it shaped to fit its own history after the fact? It is not one of AlgoChef's five scores. It never changes CSI, Profitability, Risk, Confidence, or Health, and it never gates anything — it runs alongside them as an independent read on how the backtest was built.

What This Check Answers

A backtest can look excellent by every headline number — a strong profit factor, a strong risk-adjusted return, a tidy win rate — and still have been shaped by hindsight: a date filter, a rule tweak, or years of trial and error nudged until the equity curve looked right. Summary statistics alone often can't tell a genuine edge apart from a well-disguised one, because both can produce the same clean numbers. Overfit Likelihood looks past those summary numbers at the structure of the trade-by-trade history — the shape and timing of the trades themselves — for the fingerprints a hindsight-shaped rule set tends to leave behind.

The Five Checks

Each strategy is run through five independent checks. A check "flags" when it finds a pattern consistent with hindsight fitting; it "passes" when it doesn't.

Luck vs edge. This check looks at whether the strategy's edge is large enough, given how many trades stand behind it, to be told apart from noise — and, when you've told AlgoChef how the strategy was built, whether an edge this size is the kind that would be expected to turn up simply by trying enough variations. A flag means the result is explainable by chance, or by how many attempts the build process likely made. A sample too thin to tell an edge apart from noise flags on its own, whatever the build method — that part of the check doesn't wait on your answer. What your answer changes is the rest of it: without it, this check reports rather than judges, showing how the edge would read under a few different assumptions instead of picking one.

Edge trajectory. This check compares the per-trade edge in the earliest stretch of the backtest against the edge in the most recent stretch. A real edge tends to hold roughly steady, or fade a little, over a long history. A flag means the edge is either collapsing sharply in the later blocks — the rules stopped working once they ran out of familiar data — or getting sharper over time, which is what a rule set looks like after it's been polished against the most recent data. It can't tell you why the trajectory moved, only that it did.

Sat-out periods. This check looks for stretches where the strategy held a single position far longer than its own normal holding period, or sat out of the market for an unusually long gap between trades. A flag means at least one hold or gap stands out sharply against the strategy's own typical cadence — the kind of pattern a date or equity-curve filter written with the benefit of hindsight tends to leave. It can't distinguish a filter built to dodge a known bad stretch from a genuinely unusual market condition the strategy happened to sit out.

Plausibility vs live records. This check weighs a strategy's risk-adjusted return, profit factor, and drawdown together against the envelope of what live, multi-year trading track records typically achieve. A flag means the numbers sit outside that envelope on several of those measures at once. It can't prove that a strategy with plausible numbers is genuine — only that an implausible one deserves more scrutiny before it's trusted.

Profit concentration. This check asks how much of the total profit survives once the handful of best trades, or the single best few months, are removed. A flag means the edge depends heavily on a small number of standout trades or one short lucky stretch, rather than being spread across the strategy's full history. It can't tell whether those standout trades reflect a real, occasional large win or an artifact of how the backtest was built.

"I Don't Know" and the Three Build Methods

Every strategy carries an answer to "How was this strategy built?" — with "I don't know" as the default. The other three options are: hand-built with little or no optimization, optimized parameters on an otherwise fixed rule set, and machine-generated by tools that evaluate large numbers of candidate rule sets automatically.

This answer affects exactly one of the five checks: luck vs edge. Left on "I don't know," that check shows how the edge would read under all three build methods side by side, without picking one, and none of the three moves the verdict. Declaring an actual build method lets the check weigh the edge against how many attempts a process like that typically makes before landing on a result — which can clear a thin-looking edge that was genuinely found with little searching, or flag a strong-looking one that more plausibly emerged from trying many variations. Declaring a build method never changes what the other four checks see in the trade history itself, and it can only ever help the luck-vs-edge check reach a more informed answer — it can't override a fingerprint the other four checks have already found.

Why Statistics Alone Aren't Enough

One of the strategies used to build this check is a 19-year daily backtest with excellent headline statistics — a strong profit factor, a strong risk-adjusted return, a high win rate. Judged purely on the numbers, it looks exactly like the kind of strategy this platform exists to validate. But its trade-by-trade structure tells a different story: the strategy sits completely out of the market for one stretch far longer than its own normal pattern, and its headline numbers land outside the envelope of what live, multi-year track records typically achieve on several of those measures at once. Overfit Likelihood reads this strategy as High — not because any single statistic is wrong, but because the shape of its backtest carries the fingerprints of a rule set built around what already happened, rather than discovered as something that would have worked without knowing the answer in advance. This is the reason the check exists: a strategy can pass every other score on this platform and still have been built by looking at the answer first.

What the Tiers Mean and What to Do

What This Check Does Not Do

• It's not one of the five scores. Overfit Likelihood never changes CSI, Profitability, Risk, Confidence, or Health — it's computed and shown independently, alongside them.
• It never gates anything. No strategy is disabled, hidden, or blocked from use because of what this check finds. The tier is information, not a lock.
• The build-method answer has one job. It can rule "found purely by luck" in or out for the luck-vs-edge check. It can't clear a strategy that the other four checks have already flagged for a structural reason in its trade history — telling AlgoChef a strategy was hand-built doesn't erase a sat-out period or a concentrated profit history that's actually there.

A Word on Limits

These five checks look for the structural fingerprints fitting tends to leave behind — they don't inspect the underlying rule set itself, and no single check is proof on its own. A strategy that was genuinely fitted with hindsight will usually read Elevated or High, but not always High; some hindsight fitting leaves only a partial trace. On the other side, a strategy with a real but early-stage edge can also read Elevated, because with too few trades behind it, a thin real edge is statistically indistinguishable from a lucky one. Read this check alongside the Confidence Score, which speaks directly to whether the sample size is large enough to trust either reading.