Trading / Research diagnostics

20260909T034008.157793Z-98aa0646
Local artifact · no external services

What survives
the strategy search?

Backtest overfitting diagnostic: compare the selected Sharpe ratio with what repeated searching can produce under the declared assumptions.

Synthetic demonstration

Selected training Sharpe

1.71

Annualized · selected in sample

Selected holdout Sharpe

1.24

Annualized · informational only

Deflated Sharpe diagnostic

50.1%

Declared screen ≥ 95.0%

Expected maximum Sharpe

1.71

Annualized · selection-adjusted benchmark

The search changes the benchmark

Synthetic zero-drift prices · educational demonstration

Distribution of trial training annualized Sharpe ratios1000 finite trials. Teal bars show training Sharpe counts. A pink line marks the selected trial and an amber dashed line marks the expected maximum under the declared selection assumptions.0244871950 trials2 trials1 trials0 trials6 trials14 trials22 trials21 trials33 trials45 trials59 trials69 trials79 trials76 trials95 trials74 trials78 trials68 trials66 trials44 trials40 trials27 trials26 trials20 trials10 trials8 trials10 trials2 trials4 trials0 trials1 trials0 trialsSelected trial: 1.71Expected maximum: 1.71-1.6-0.9-0.20.51.21.9Annualized Sharpe · training window
Trial distributionSelected trialExpected maximum

Below the declared diagnostic threshold

This screen applies to the reported training DSR. Passing is not a strategy-validation decision.

Selection contract

Highest training per-period sample Sharpe; lexical name tie break. Holdout never selects the winner.

Selected trial
random_00729
Declared trials
1,000
Evaluated trials
1,000
Effective independent trials
1,000.00
Sharpe dispersion · per period
0.03315
Periods per year
252
Training observations
756
Holdout observations
252
Training interval
2000-01-02T00:00:00+00:00 → 2002-01-26T00:00:00+00:00
Holdout interval
2002-01-27T00:00:00+00:00 → 2002-10-05T00:00:00+00:00

Follow the selected path

Compounded excess-return indices, each starting at 1; these are not account wealth. The holdout is reported after selection and is not used to rank trials.

Training · in sample

Training compounded excess-return indices, each starting at 1Teal shows the selected trial; the dashed gray line shows the supplied baseline. Each window resets its excess-return index to 1; these are not account wealth paths.0.731.121.521.912.31BaselineSelected trialStartEndObservation order · compounded excess-return index

Holdout · out of sample within this run

Holdout compounded excess-return indices, each starting at 1Teal shows the selected trial; the dashed gray line shows the supplied baseline. Each window resets its excess-return index to 1; these are not account wealth paths.0.870.971.061.151.24BaselineSelected trialStartEndObservation order · compounded excess-return index
Selected trialSupplied baseline

Out of sample describes this run's split. Prior use of imported data cannot be verified here. Display paths are sampled to approximately 1,200 points when longer; full returns remain in the run exports.

Selected return profile

Sample moments expose assumptions a single Sharpe ratio hides.

MetricTrainingHoldout
Sharpe · per period0.10800.0783
Mean return · per period0.0010800.000788
Return standard deviation0.0100010.010062
Skewness0.0320.231
Kurtosis · non-excess2.9722.719
Lag-1 autocorrelation0.0220.022
PSR · zero-Sharpe benchmark99.8%—

How much searching?

Sensitivity to assumed independent trial counts; the same Sharpe dispersion and selected return moments are held fixed.

Assumed trialsExpected max SR · ann.DSR
1.000.0099.8%
2.000.2799.4%
10.000.8393.7%
100.001.3374.6%
1,000.001.7150.1%

These scenarios do not estimate independence from strategy correlation or recover unlogged experiments.

Simple comparators

Annualized Sharpes. An undefined Sharpe, including a constant-return cash series, is shown as —.

BaselineTrainingHoldout
First predeclared candidate: random_000010.011.16
Buy and hold0.280.40
Cash: zero excess return (zero variance; Sharpe undefined)——

Inspect the full search

Training ranks determine the selection. Use the column buttons to inspect results; changing the display does not change the selected trial.

1,000 reported trials
random_00729Selected on training data1.711.2499.8%50.1%
random_006571.520.5899.6%36.6%
random_000471.48-0.3899.5%34.3%
random_000611.46-0.4999.4%33.3%
random_008661.450.6599.4%32.3%
random_007301.440.3399.4%32.1%
random_009391.34-2.5098.9%25.9%
random_000811.330.3898.9%25.5%
random_002781.320.2598.9%24.6%
random_001071.31-0.3598.8%24.5%
random_005851.31-1.1198.8%24.4%
random_007741.30-0.2398.8%24.0%
random_000311.30-2.5898.8%23.9%
random_007431.27-0.6198.7%22.1%
random_007541.27-1.8898.6%22.1%
random_001701.240.0698.4%20.5%
random_000171.23-1.1898.3%20.0%
random_006961.22-0.2498.2%19.7%
random_007721.220.8498.3%19.5%
random_001571.18-1.0597.9%17.8%
random_009971.15-0.5397.7%16.7%
random_001451.15-0.8297.7%16.4%
random_006381.14-0.4497.6%16.4%
random_005951.14-0.0497.6%15.9%
random_002411.120.3097.4%15.4%
random_009601.10-0.0397.2%14.6%
random_002801.100.4197.1%14.5%
random_008861.080.6596.9%13.5%
random_000741.07-0.9796.8%13.4%
random_008171.07-1.8196.8%13.3%
random_002641.06-0.2696.7%12.9%
random_007501.060.7296.6%12.9%
random_006831.060.9496.6%12.8%
random_002241.041.1396.3%12.1%
random_001321.02-0.5696.2%11.5%
random_006740.97-0.6895.3%10.1%
random_008900.970.1895.4%9.8%
random_007730.96-0.3795.2%9.7%
random_005130.961.9195.2%9.8%
random_006220.96-1.5095.2%9.6%
random_004090.95-1.2194.9%9.3%
random_009570.95-1.4894.9%9.3%
random_003350.95-0.0695.0%9.2%
random_004100.94-0.6094.8%9.1%
random_005860.940.9494.8%9.1%
random_005580.931.8294.5%8.7%
random_001460.930.8794.4%8.8%
random_004600.910.6394.2%8.4%
random_003970.91-0.1494.4%8.2%
random_004170.911.6194.3%8.3%
random_008820.910.0494.3%8.3%
random_001890.91-0.1094.1%8.3%
random_009820.900.3994.2%7.9%
random_001210.900.3394.0%7.9%
random_006410.900.5393.9%7.9%
random_000720.892.5993.9%7.8%
random_003170.880.0493.6%7.7%
random_003920.880.2193.7%7.5%
random_008790.880.2393.7%7.5%
random_001960.88-1.3493.5%7.4%
random_006370.871.5793.4%7.4%
random_000850.861.0193.2%7.0%
random_001250.861.6093.2%6.9%
random_003790.85-0.7993.1%6.8%
random_008050.85-0.2093.0%6.8%
random_000160.85-0.1492.9%6.6%
random_004500.85-0.4892.8%6.7%
random_006720.84-0.9492.6%6.5%
random_001300.830.8492.5%6.4%
random_002120.830.8992.4%6.3%
random_004990.82-1.0092.2%6.2%
random_004800.822.0392.2%6.1%
random_005340.820.0692.1%6.0%
random_001200.81-2.5392.0%6.0%
random_007220.810.6992.0%5.9%
random_001180.81-1.0492.0%5.8%
random_000560.81-0.8291.8%5.9%
random_001680.80-0.4191.7%5.7%
random_006030.800.6391.6%5.7%
random_005610.80-0.3591.6%5.7%
random_004830.790.3091.5%5.5%
random_006270.77-0.8391.0%5.2%
random_004140.771.4891.0%5.2%
random_003500.77-0.8390.9%5.1%
random_007820.770.0790.8%5.0%
random_001690.760.4490.4%4.9%
random_008340.750.4490.3%4.9%
random_006650.750.3690.3%4.7%
random_001540.75-0.9190.1%4.7%
random_002730.74-0.4890.1%4.7%
random_002680.74-1.6790.1%4.7%
random_001830.74-1.4390.1%4.7%
random_007280.740.8489.9%4.6%
random_001620.74-0.8889.9%4.6%
random_007560.730.8689.8%4.4%
random_002150.73-0.8689.6%4.5%
random_004450.73-0.5589.5%4.4%
random_004870.72-0.0489.4%4.3%
random_000520.720.5689.3%4.3%
random_001120.71-0.5589.1%4.2%

Showing first 100 of 1000 trials

Sharpe columns are annualized. PSR and DSR use training per-period returns. Trial-by-trial diagnostics are descriptive; inspecting them introduces further selection choices.

Interpretation and limits

Run-specific warnings