Evaluating Backtest Claims: Sharpe vs. Deflated Sharpe Ratio

Note on figures cited below. Some statistics in this article come from the V12 universe cohort (227,594 BUY filings, period through 2024). Since 2025-01-01 the live out-of-sample BUY universe (28 markets) shows a 52.96 percent win rate and a +8.59 percent mean 90-day return on 20,688 filings. See methodology for the current numbers and the STRATEGY_PROOF block for the frozen audit reference.

A backtest with a handsome Sharpe and no accounting for selection bias is not evidence, it is marketing with decimals.

Sharpe vs DSR, how to read backtest claims with our published numbers

Backtests are full of confident numbers produced under suspiciously forgiving conditions. The Sharpe ratio is the usual headline act, tidy, familiar, and often granted more authority than it deserves. It is useful, but only in the same way a blood pressure reading is useful. One number can matter a lot, and still be a poor summary of the whole patient.

For a publication that covers insider transactions and publishes performance pages, this matters. Readers should be able to distinguish between a strategy that is genuinely robust and one that merely survived a tournament of parameter fiddling. The distinction is not academic. It is the difference between evidence and overfit theatre.

The Deflated Sharpe Ratio, or DSR, is one of the better tools for this job. Introduced by David Bailey and Marcos López de Prado in 2014, it asks a simple and slightly impolite question: given the number of strategies likely tried, the shape of returns, and the sample length, how surprising is this Sharpe really? Many backtests do not enjoy being asked.

The Sharpe ratio is useful, and dangerously easy to oversell

The Sharpe ratio remains standard because it answers a reasonable first question: how much excess return did a strategy deliver per unit of volatility? Higher is better, all else equal. It is simple enough to compare across strategies, and common enough that readers know roughly what counts as weak, decent, or suspiciously pretty.

That last category deserves emphasis.

A Sharpe of 1.2 can be impressive. It can also be the best survivor from 300 variants, each with slightly different holding periods, universe filters, liquidity cuts, and event windows. By the time the winning specification reaches the page, the failed cousins have been quietly buried. The published Sharpe then reflects not just the strategy, but the search process that found it.

What Sharpe does measure

At its best, Sharpe is a compact summary of realised mean excess return relative to realised volatility. It is useful for:

• comparing variants on the same dataset,
• checking whether a return stream is mostly noise,
• ranking strategies before deeper due diligence.

If two strategies are built honestly, on comparable data, with similar turnover and implementation assumptions, Sharpe is a decent starting point.

What Sharpe does not measure

Sharpe does not know:

• how many ideas were tried before this one,
• whether the return distribution is skewed or fat-tailed,
• whether the strategy depends on one odd period,
• whether transaction costs were estimated generously,
• whether the researcher stopped when the chart looked nice.

This is the first practical rule for reading any backtest claim: a Sharpe ratio without context is a teaser, not a conclusion.

Why insider-trading strategies are especially vulnerable

Event-driven strategies built on insider filings are fertile ground for accidental overfitting. There are many knobs to turn:

• buy only, or buy and sell,
• CEO only, C-suite only, or all insiders,
• cluster trades vs single trades,
• value thresholds,
• reporting lag assumptions,
• holding periods from a few days to several months,
• country-specific disclosure rules,
• liquidity and market-cap filters.

Each choice can be justified. That is the problem. A large research space makes it easy to discover something that looks significant by chance alone. If the article reports only the winner, the Sharpe is flattered by hidden multiple testing.

Deflated Sharpe Ratio is the backtest's awkward audit

The Deflated Sharpe Ratio was designed to address a problem the industry had become very good at ignoring. Researchers were presenting attractive Sharpe ratios from large families of tested strategies, often under non-normal return distributions, and readers were treating those Sharpe ratios as if they came from a single clean hypothesis test.

Bailey and López de Prado proposed a correction that accounts for three things at once:

1. the observed Sharpe ratio,
2. the number of observations and properties of the return distribution, especially skewness and kurtosis,
3. the number of trials, or effectively independent strategy variants, tested.

The output is not just a prettier ratio. It is a probability-based assessment of whether the observed Sharpe is likely to be genuinely above a benchmark once the data-mining problem is acknowledged.

The intuition behind DSR

Suppose you test one strategy and obtain a Sharpe of 1.0 over a long sample with fairly well-behaved returns. That may be interesting.

Now suppose you test 500 variants and the best one has a Sharpe of 1.0. Less interesting. In a big enough search, someone was going to look clever.

DSR asks whether the observed Sharpe exceeds the Sharpe one would expect from the best of many lucky draws. It then adjusts for finite sample effects and non-normality. The result is a more sceptical reading of the same performance record.

That scepticism is usually healthy.

Why non-normality matters

Many event-driven strategies, including those based on insider filings, do not produce tidy Gaussian return streams. They can be:

• positively or negatively skewed,
• exposed to clustered drawdowns,
• dependent on sparse events,
• sensitive to liquidity shocks.

A plain Sharpe ratio compresses all of that into mean and variance. DSR explicitly incorporates skewness and kurtosis in the significance adjustment. This matters because a strategy with lumpy returns can look better in Sharpe terms than its true statistical reliability warrants.

DSR is not magic, but it is harder to game accidentally

No metric can rescue a fundamentally poor research design. If costs are omitted, timestamps are wrong, or the signal leaks future information, DSR will not save the paper. But it does raise the burden of proof. It forces the author to admit that the strategy did not emerge from a vacuum.

That alone is progress.

Multiple testing is the quiet villain in most performance pages

The phrase sounds technical, but the mechanism is ordinary. If you try enough things, one of them will work by chance. Finance research is especially exposed because the design space is huge and the incentives are not subtle.

A publication wants a clean story. A fund wants a compelling deck. A researcher wants to avoid writing, "we tested 84 variants and most were rubbish." The result is a familiar asymmetry. Successes are visible, failures are compost.

The family of hidden trials

When we evaluate insider-transaction strategies, the true number of trials is often larger than the formal count of models. It includes:

• alternative event definitions,
• threshold tuning,
• different rebalance frequencies,
• subperiod exclusions,
• benchmark choices,
• winsorisation rules,
• cost assumptions,
• regional subsets.

Even the decision to report one chart and not another is part of the selection process. The effective number of independent trials can be difficult to estimate, but pretending it is one is plainly wrong.

Why p-values are not enough

Traditional significance tests can help, but they often assume a single pre-specified hypothesis. Backtests rarely deserve that assumption. By the time a strategy is published, the path to publication may have involved many informal checks and revisions. The nominal p-value then understates the true false-discovery risk.

DSR was built for this reality. It is not the only correction available, but it is one of the most practical for strategy evaluation because it speaks the language practitioners already use, Sharpe.

The standard abuse pattern

The pattern is almost too neat:

1. Start with a broad event universe.
2. Try several filters.
3. Keep the strongest subgroup.
4. Tune the holding period.
5. Adjust cost assumptions until the net line remains attractive.
6. Publish the best Sharpe as if it were the original hypothesis.

This is not necessarily fraud. Often it is just ordinary research behaviour under deadline pressure. The effect on readers is the same.

What we should publish on /performance, and what we should stop implying

If a performance page exists to inform rather than impress, it should report enough to let a reader judge robustness. That means the headline metric cannot be the whole story.

Our published numbers for this article are not available from the live data block, so the specific Sharpe and DSR values are n/a. Fine. Better to say n/a than to improvise a number and hope no one notices. The reporting framework still matters, because it determines how future numbers should appear.

The minimum set of metrics

A sensible performance page for insider-strategy backtests should include:

• annualised return,
• annualised volatility,
• Sharpe ratio,
• maximum drawdown,
• turnover,
• estimated transaction costs,
• sample start and end dates,
• number of events or trades,
• benchmark used,
• DSR or an equivalent multiple-testing-aware significance check.

Without these, a Sharpe ratio floats free of context. A strategy with a Sharpe of 1.1 and brutal turnover may be less investable than a strategy with a Sharpe of 0.8 and lower implementation drag.

The minimum set of disclosures

Metrics are not enough. We should also disclose:

• whether the specification was pre-registered or selected after exploration,
• the approximate number of tested variants,
• whether the reported result is in-sample, out-of-sample, or walk-forward,
• how filing timestamps and publication lags are handled,
• whether delisted securities are included,
• whether returns are value-weighted or equal-weighted.

These details are not decorative. In insider-trading research, timestamp handling alone can decide whether a signal is plausible or fictional.

What DSR should look like in practice

A clean implementation does not need to drown readers in notation. It can present:

• observed Sharpe,
• assumed number of trials or effective independent tests,
• sample length,
• skewness and kurtosis adjustment,
• resulting DSR or probability of genuine skill.

If trial count is uncertain, report a range. For example, "DSR remains above threshold assuming 20 to 50 effective trials, but falls below under 100." That is more informative than a single polished number with no assumptions attached.

Why we should resist the temptation to simplify too far

Performance pages often simplify because readers like clean dashboards. Sensible. But simplification becomes distortion when uncertainty is hidden. A strategy can have a respectable raw Sharpe and a weak DSR. That is not a contradiction. It is a clue that the result may owe more to search than to signal.

Readers can handle this. They already understand that gross and net returns differ. They can also understand that raw and deflated Sharpe differ.

Reading our own published numbers without flattering ourselves

Because no live performance data were fetched for this article, we cannot discuss a specific current Sharpe or DSR from our /performance page. The correct value is n/a. There is still a useful exercise here: how should a reader interpret the numbers once published?

If the raw Sharpe is high and DSR is also strong

This is the best case. It suggests the strategy's performance remains notable even after accounting for multiple testing and return-shape effects. It does not prove future success, because markets are not obliged to repeat themselves, but it does indicate the result is less likely to be a statistical mirage.

A robust interpretation would still ask:

• Was there a clean out-of-sample period?
• Are costs realistic?
• Is the signal economically intuitive?
• Does the result survive across regions and subperiods?

If the raw Sharpe is high and DSR is weak

This is common, and more revealing than embarrassing. It usually means one of three things:

1. too many variants were explored relative to the sample size,
2. returns are too irregular for the observed Sharpe to be persuasive,
3. the effect is concentrated in a narrow slice of the data.

In that case, the right editorial stance is not to bury the strategy. It is to label it correctly. "Interesting, but not yet robust" is a respectable conclusion. Better that than pretending a beauty contest winner is a law of nature.

If both are modest

Then the strategy may still have value as a component signal, a risk diversifier, or a research lead. Not every useful strategy deserves a heroic standalone page. Some belong in composites, where modest standalone edges combine into something sturdier.

If the strategy only works after many filters

Treat it as a hypothesis generator, not a product. This is especially true in insider-transaction datasets, where subgroup slicing can create persuasive narratives around tiny samples. "Micro-cap French industrial CEOs buying after clustered declines" may indeed have worked once. It may also be the statistical equivalent of finding Jesus in the toast.

A practical framework for readers, editors, and researchers

The point of all this is not to replace one fetish number with another. DSR is helpful because it forces discipline, not because it grants certainty. A good reporting standard should make weak evidence look weak and strong evidence harder to fake.

For readers: three questions to ask immediately

When you see a backtest claim, ask:

1. How many variants were probably tried before this one?
2. Is there any multiple-testing-aware metric, such as DSR?
3. Are the implementation assumptions credible?

If the answer to the second is no, and the first is obviously "many", downgrade your confidence sharply.

For editors: what to require before publication

An editorial checklist should require:

• a raw Sharpe ratio,
• a deflated significance check,
• sample length and event count,
• explicit cost assumptions,
• at least one robustness test,
• a note on selection process.

This is not bureaucratic fussiness. It is the minimum needed to stop publishing false precision.

For researchers: how to improve the odds that a result survives scrutiny

Pre-specify where possible. Keep a research log of tested variants. Reserve a true out-of-sample period. Use realistic execution assumptions. Report negative findings internally. Most importantly, assume your favourite result is guilty until proven otherwise.

That may sound severe. It is also cheaper than discovering, after launch, that the alpha was a formatting choice.

The wider lesson for insider-transaction research

Insider filings are compelling because the underlying economic story is compelling. Managers know their firms better than outside investors. Sometimes they buy for reasons that matter. Sometimes they sell for reasons that do not. The signal is plausible, but plausibility is not immunity from overfitting.

A large dataset, and ours spans 162,000 filings according to the brief, increases opportunity as well as temptation. More observations allow better inference. They also allow more slicing, more subgroup stories, and more accidental discoveries. Scale is helpful. It is not absolution.

Why this matters more now

Performance pages travel. A chart published as research can be screenshotted into marketing by lunch. Once a Sharpe ratio is detached from its caveats, it acquires a false dignity. The remedy is not to stop publishing performance. It is to publish it with enough structure that caveats travel with the claim.

What a better standard would look like

A mature standard for insider-strategy reporting would make these items routine:

• raw Sharpe and DSR side by side,
• trial-count assumption stated plainly,
• out-of-sample result separated from in-sample result,
• net-of-cost performance prioritised over gross,
• event-timestamp methodology documented.

That would not eliminate disagreement. It would improve the quality of disagreement, which is usually the best one can do in empirical finance.

The open question we should keep asking

The hardest part of DSR in practice is estimating the effective number of independent trials. Researchers know the literal number of variants tested, but not all variants are distinct in a statistical sense. A holding period of 20 days and 21 days are not independent ideas. Nor are "CEO only" and "C-suite" if the overlap is large.

This is where judgement enters, and where transparency matters most. A publication should not pretend the estimate is exact. It should explain the range and show how sensitive the conclusion is to that range.

That is the adult version of quant reporting. Less glamour, more credibility.

The next step is concrete: on our /performance pages, pair every published Sharpe with a stated trial-count assumption and a deflated significance check, then revisit any legacy strategy whose headline number looks sturdy only because no one asked how many attempts it took to find it.

Editorial note, this article was assembled without live web research (Grok unavailable in this generation pass). Evergreen sources are cited above; numerical claims are pulled from our own database snapshot as of 2026-05-17.

Insiders Trades backtest cohort

The analysis of our backtest data reveals critical insights into the performance of trading strategies across a substantial sample size. With 220,691 priced trades in the BUY cohort spanning 17 markets, the mean 90-day return stands at -1.70%, accompanied by a win rate of 41.82%. The annualised Sharpe ratio for this cohort is -0.08, mirroring the aggregate findings from the broader global universe, which includes 227,594 BUY filings and also reports a mean 90-day return of -1.70% and an aggregate Sharpe ratio of -0.08. These figures underscore the challenges faced by traders in achieving positive returns and highlight the relevance of both the Sharpe and deflated Sharpe ratios in evaluating strategy performance. However, it is essential to consider that these results are based on a specific sample size and a 90-day time horizon, which may influence the robustness of the conclusions drawn.

Go deeper

This article is part of our working with insider data series. Adjacent reading:

• Building a quant model with our REST API: reproduce the numbers yourself
• Survivorship bias in insider studies: the other silent backtest killer
• The composite score audit: the weights behind the published strategy
• How to read an insider transaction: the signal these statistics measure

The current out-of-sample results, test window and audit trail live on the performance page and the methodology page.

Last reviewed · 2026-05-18 · By Simon Azoulay · Sources, SEC EDGAR, AMF BDIF, and 28 other regulators.