Deflated Sharpe Ratio Explained: Multiple Trials and Backtest Selection

A researcher may try many signal definitions, lookbacks, entry thresholds, universes, holding periods, and cost settings, then report the version with the highest Sharpe ratio. Even if every candidate has zero true skill, the maximum estimated Sharpe tends to rise as the search gets wider. Choosing that maximum makes the reported result different from a single strategy that was specified before looking at the data.

Bailey and López de Prado’s Deflated Sharpe Ratio addresses two sources of inflation in a selected Sharpe: multiple testing and non-normal returns. The first raises the benchmark that the result must clear; the second changes the estimated uncertainty of the Sharpe. The original paper describes DSR as a Probabilistic Sharpe Ratio evaluated against a selection-adjusted threshold. For the underlying reward-to-variability measure, see Sharpe’s original explanation.

That framing prevents a common interpretation error. DSR does not mechanically subtract a fixed penalty from the reported Sharpe and return a new performance ratio. It estimates how strongly the observed Sharpe exceeds a hurdle tied to the search, sample length, and return moments. A result can have an unchanged point estimate and a lower DSR because its evidence is weaker.

Suppose every tested rule has the same true Sharpe, but each estimate varies because it uses a finite sample. Selecting the largest estimate also selects a favorable measurement error. This is the winner’s curse: the selected estimate is expected to regress toward the group’s typical value when evaluated on new data.

The number of trials matters, but so does how different their estimates are. If candidates produce nearly identical returns, their estimated Sharpes are strongly related and do not provide as many independent chances to find a lucky maximum as an equally large set of unrelated candidates. The raw number of rows in a parameter grid is therefore not automatically the number of independent trials.

The search record can extend beyond a formal grid. A manually changed threshold, a discarded market, or a revised cost assumption may have influenced which result survived. A DSR computed from only the final candidates cannot account for experiments that were not recorded or supplied to the calculation.

The Probabilistic Sharpe Ratio (PSR) estimates whether a sample Sharpe exceeds a chosen benchmark, accounting for the number of return observations and estimated skewness and kurtosis. DSR uses the same basic calculation but chooses a benchmark that reflects the expected maximum Sharpe from the research search.

For one common PSR approximation, the selection-adjusted calculation is:

$$ \mathrm{DSR}=\Phi\!\left( \frac{(\widehat{SR}-SR_0)\sqrt{T-1}} {\sqrt{1-\widehat{\gamma}_3\widehat{SR} +\frac{\widehat{\gamma}_4-1}{4}\widehat{SR}^{\,2}}} \right) $$

Here, $\Phi$ is the standard normal cumulative distribution function, $\widehat{SR}$ is the observed Sharpe per return observation, $SR_0$ is the selection-adjusted hurdle in the same units, and $T$ is the number of observations. $\widehat{\gamma}_3$ is the sample skewness. $\widehat{\gamma}_4$ is Pearson kurtosis, where a normal distribution has kurtosis 3. The denominator reflects how skewness and kurtosis change Sharpe-estimation uncertainty.

This expression is sensitive to convention. Use excess returns at one frequency, calculate the Sharpe and the hurdle at that same frequency, and define kurtosis consistently. Annualizing only one of those quantities changes the comparison. The familiar formula is also based on assumptions about the return observations; serial dependence needs its own treatment rather than a silent substitution into $T$.

For $N$ independent Sharpe estimates with an approximately normal distribution, Bailey and López de Prado use an extreme-value approximation for the expected maximum:

$$ SR_0 \approx \mu_{SR}+\sigma_{SR} \left[(1-\gamma_E)\Phi^{-1}\!\left(1-\frac{1}{N}\right) +\gamma_E\Phi^{-1}\!\left(1-\frac{1}{Ne}\right)\right] $$

$\mu_{SR}$ and $\sigma_{SR}$ describe the mean and standard deviation of the trial Sharpe estimates, $\Phi^{-1}$ is the standard normal quantile function, $e$ is Euler’s number, and $\gamma_E \approx 0.5772$ is the Euler–Mascheroni constant. The expression estimates how high the best estimate can be from selection alone under the chosen trial model. It is not a universal threshold for every strategy search.

When candidate results are correlated, treating every variation as independent can overstate the effective number of trials. The paper discusses estimating an independent-trial count from candidate dependence, but any such estimate is another modeling choice. Correlation is only one form of dependence, and an estimate based on many candidates and a short return history can be unstable. Report the raw candidate count, the method used for any effective count, and sensitivity to plausible alternatives.

Assume a deliberately simplified search with 20 independent candidate strategies. Under the null, take the mean of their Sharpe estimates as 0 and their standard deviation as 0.20 in monthly units. The expected-maximum approximation gives a selection hurdle of about $SR_0=0.380$ per month. These inputs are assumptions for arithmetic, not observations from a market or a recommended benchmark.

Now suppose one selected strategy has 60 monthly excess-return observations, an estimated monthly Sharpe of 0.50, sample skewness 0, and Pearson kurtosis 3. The PSR component compares 0.50 with 0.380 rather than with zero:

$$ z=\frac{(0.50-0.380)\sqrt{59}}{\sqrt{1+((3-1)/4)(0.50)^2}} \approx 0.868,\qquad \mathrm{DSR}=\Phi(z)\approx 0.807 $$

The resulting DSR is about 80.7% under these simplifying assumptions. The value is not an 80.7% probability that the strategy will make money, nor a forecast of next month’s return. It is the method’s estimated evidence that the underlying Sharpe exceeds the chosen selection hurdle, conditional on the model and inputs. A different trial count, dependence estimate, sample, or return shape can change the result.

Sensitivity to the trial-estimate spread is material. If only the assumed standard deviation of candidate Sharpe estimates changes from 0.20 to 0.10, the same 20-trial search produces a hurdle of about 0.190 and DSR of about 98.8%. If the standard deviation is 0.30, the hurdle is about 0.570 and DSR is about 30.6%. The observed Sharpe of 0.50, 60 observations, skewness, and kurtosis stay fixed. This hypothetical sensitivity shows why the method for estimating the trial spread and candidate dependence matters; the values still use the same monthly units and simplified model.

Two strategies can report the same sample Sharpe and sample length while having different return distributions. A pronounced left tail, asymmetry, or unusually frequent extreme observations can change the sampling uncertainty represented in the PSR denominator. The adjustment is calculated from the measured moments; it does not label every non-normal distribution as equally harmful or guarantee that a few sample moments fully describe tail behavior.

The arithmetic also depends on what counts as one observation. Daily, weekly, and monthly data produce different values for $T$, Sharpe, skewness, and kurtosis. Overlapping holding-period returns can make adjacent rows dependent. A strategy with smoothed or stale marks may appear less variable than its underlying economic risk. Describe the sampling frequency and construction instead of treating the same annualized Sharpe as sufficient input.

PBO uses combinatorially symmetric cross-validation to ask how often the in-sample winner ranks at or below the candidate median on complementary blocks. Bailey and coauthors formalize that selection diagnostic in their original PBO paper. DSR estimates whether a selected Sharpe exceeds a hurdle adjusted for the search and non-normality. Their outputs and resampling assumptions differ; one cannot be substituted for the other. See the guides to [PBO and CSCV](/en/learn/probability-of-backtest-overfitting-cscv-explained) and [multiple testing in finance](/en/learn/multiple-testing-false-discovery-rate-finance-explained).

White’s original Reality Check paper uses a bootstrap to test whether the best rule in a candidate family beats a specified benchmark after accounting for data snooping. Its question is benchmark-relative, while DSR applies a Sharpe-based hurdle. A chronological walk-forward or final holdout asks how a frozen process behaves on later periods. These approaches can complement one another because they evaluate different parts of the research claim.

A useful report identifies the candidate universe, every documented decision that affected selection, the raw number of trials, any effective-trial method, the return series, sample length, frequency, Sharpe definition, skewness, kurtosis convention, and selection-adjusted hurdle. State whether returns are gross or net of spread, commissions, market impact, funding, borrow, and other costs. Show the observed Sharpe and DSR together so a reader can see what changed.

The conventional formula assumes a sampling model that may not fit serially correlated observations. Lo’s work on Sharpe-ratio statistics explains why time aggregation and dependence affect Sharpe inference. If returns overlap or show serial correlation, use an inference procedure that addresses that structure and explain its assumptions; multiplying a monthly Sharpe by $\sqrt{12}$ does not correct it. For time-aware evaluation, also see the guide to [expanding and rolling walk-forward windows](/en/learn/walk-forward-validation-expanding-vs-rolling-windows-explained).

DSR cannot discover an undocumented search, remove look-ahead leakage, make survivorship-biased data representative, validate fills, estimate capacity, or guarantee stability through a new regime. It does not replace economic reasoning or later chronological evidence. Treat it as one documented diagnostic within a research process, and keep the candidate record and data pipeline reproducible.