The Deflated Sharpe Ratio, read closely

Toss a fair coin ten times and you might see {+,+,+,+,+,−,−,−,−,−}. A researcher stares at it and “discovers” a rule: bet heads for the first five tosses, tails for the last five. On this sample the rule is perfect. On the next ten tosses it is a coin flip. The rule never had power over the future; it was fitted to the way the noise happened to fall. This is backtest overfitting, and López de Prado's point is that modern compute makes it nearly unavoidable: a quant team can now try millions of parameter combinations on the same price history, and something will always look brilliant in sample.

The statistical name for the trap is multiple testing. Run one test at a 5% significance level and you accept a 5% chance of a false positive. Run twenty independent tests and the chance that at least one looks significant by luck is 64.2%. Keep only the winner and report it as a single result, and you have committed selection bias — the paper's core charge against the finance literature. The most important number in any backtest, the authors argue, is the one almost never disclosed: how many trials it took to find.

Here is the idea made exact. Suppose a class of strategies genuinely has zero skill, so each trial's estimated Sharpe is just noise: draw N numbers from a Normal distribution with mean 0 and some variance V. What is the expected largest of those N draws? Extreme-value theory gives a clean approximation (their eq. 6),

E[max SR] ≈ √V · [ (1 − γ)·Φ⁻¹(1 − 1/N) + γ·Φ⁻¹(1 − 1/(N·e)) ]

where γ ≈ 0.5772 is the Euler–Mascheroni constant, Φ⁻¹ is the inverse Normal CDF (the quantile function), and e is Euler's number. We implement it exactly as the paper's own code snippet does — no stats library, just an inverse-Normal built from scratch. The site runs the TypeScript; the Python matches the paper's listing line-for-line.

import numpy as np
from scipy.stats import norm

EMC = 0.5772156649015329            # Euler-Mascheroni constant

def expected_max_sharpe(n_trials, var_sr):
    # Bailey & Lopez de Prado 2014, eq. 6: the expected MAXIMUM Sharpe of
    # n_trials independent zero-skill strategies, each ~N(0, var_sr).
    z = ((1 - EMC) * norm.ppf(1 - 1 / n_trials)
         + EMC     * norm.ppf(1 - 1 / (n_trials * np.e)))
    return np.sqrt(var_sr) * z       # grows without bound in n_trials

A closed form derived from extreme-value theory deserves suspicion. So we check it the honest way: draw N zero-mean Normal “Sharpes” through the site's seeded RNG, take the maximum, repeat thousands of times, and average. That Monte-Carlo mean should trace the analytic curve. It does.

The fix writes itself. If selection under N trials inflates the winner's Sharpe up to E[max SR] even with zero skill, then that is the bar a real strategy must clear — not zero. The Deflated Sharpe Ratio asks: given the winner's estimate, how confident are we the true Sharpe beats the winner's-curse threshold? It is built on the Probabilistic Sharpe Ratio, which already corrects for two other inflators — a short track record, and non-Normal returns. Real strategies tend to have negative skew (rare big losses) and fat tails (high kurtosis), and both make a given Sharpe less trustworthy than the textbook Normal case assumes.

def probabilistic_sharpe(sr, sr_star, n_obs, skew, kurt):
    # Bailey & Lopez de Prado 2012: P(true Sharpe > sr_star), using the
    # estimator's standard error under non-Normal returns (Lo 2002).
    se = (1 - skew * sr + (kurt - 1) / 4 * sr ** 2) ** 0.5
    return norm.cdf((sr - sr_star) * (n_obs - 1) ** 0.5 / se)

def deflated_sharpe(sr, n_obs, skew, kurt, n_trials, var_sr):
    # DSR = a PSR whose benchmark is the winner's-curse threshold.
    sr_star = expected_max_sharpe(n_trials, var_sr)
    return probabilistic_sharpe(sr, sr_star, n_obs, skew, kurt)

Read the DSR as a probability. Above 0.95 means: even after admitting how many strategies you tried and how ugly the returns are, there is a 95%+ chance the edge is real. Below it, the backtest is indistinguishable from the best of a pile of coin flips.

A strategist backtests a treasury-auction seasonality idea, sweeping tenors, holding periods and stop-losses. The best configuration posts an annualized Sharpe of 2.5 over five years of daily data (1250 observations). Impressive — until the investor asks the four questions the paper insists on: how many trials (N = 100), the spread of the trial Sharpes (variance 0.5 annualized), and the skew (-3) and kurtosis (10) of the returns. Feed them in and the deflation threshold climbs with N while the DSR falls through the floor:

Non-Normality is doing real work here too. If those same returns had been Normal (skew 0, kurtosis 3), the strategy would have survived up to N = 88 trials before the DSR dipped under 0.95. The fat left tail costs it roughly half its trial budget — a reminder that a Sharpe computed on lumpy, crash-prone returns was overstated before multiple testing even entered the picture.

The DSR is often sold as “the honest Sharpe.” But look again at what the table actually proves: the pass/fail line is set by N and by V, the variance of the trial Sharpes — two numbers the researcher self-reports and no referee can audit. Nudge N from 100 down to 46 and a rejected strategy becomes fundable, with not one return changed. The very statistic built to defeat selection bias can be gamed by under-reporting the selection.

There is a deeper crack, and the paper is candid about it in Appendix 3. The N in the theorem is the number of independent trials, but real strategy sweeps are heavily correlated — a hundred nearby stop-loss values are almost the same bet. The authors convert M correlated trials into an effective count with N̂ = ρ + (1 − ρ)·M: at an average trial correlation of ρ = 0.5, the analyst's M = 100 sweeps collapse to just 50.50 effective trials — which wouldraise the DSR back toward a pass. Convenient. And the estimate of ρ, the authors note, is itself often computed on more trials than the sample can support, so it “may itself be overfit.” The correction for overfitting has an overfitting problem of its own.

• Every slider is a trial. Open the gradient-boosting page and grid-search its rounds, depth, and learning rate against a noisy target. Each configuration you keep because it scored best is a trial in the sense above — and the “best” validation score you walk away with is a draw from E[max], not the truth. The DSR is the receipt for that search.
• Where does V come from? Our example takes the trial-Sharpe variance as given. In practice you estimate it from the sweep itself — but backtest optimizers deliberately concentrate trials near the winner, shrinking the apparent spread and, with it, the deflation. Measuring V on a deliberately-biased sample is its own trap.
• Beyond parametric. The DSR assumes trial Sharpes are Normal. The same authors' non-parametric alternative, the Probability of Backtest Overfitting (PBO) via combinatorial cross-validation, drops that assumption at the cost of far more data — the natural sequel to this post.

The Sharpe ratio is a single, seductive number, and this paper's gift is to remind you that a number is only as meaningful as the search that produced it. Before you believe a backtest — including your own — ask the one question the whole field trains you to skip: how many did you try before this one? Then go run a real search on the boosting page and watch the best-of-many Sharpe appear out of pure noise, exactly as the theorem promises.