Chapter 3 of 8

Test an edge without fooling yourself

Design a research protocol that measures net expectancy, adverse paths and selection bias instead of celebrating a win-rate screenshot.

4 min read · Advanced process workshop

In this chapter

Define the candidate before measuring it
Calculate net expectancy in comparable units
Treat every trial as part of the research record
Inspect uncertainty and dependence
Make forward paper testing genuinely forward
Worked example
Common mistakes
Check your understanding

Define the candidate before measuring it #

Write the instrument universe, session, data fields, setup, trigger, invalidation, exit, position sizing and execution model. Separate a research hypothesis from its implementation. “Momentum persists” is a hypothesis; “buy the next-bar open after a completed signal bar under a frozen rule” is an implementation.

Timestamp all required information. A completed candle can only be used after it closes; an indicator computed with future revisions must not appear in the earlier decision set. Use point-in-time membership for a historical equity universe, account for corporate actions, and retain delisted names where the research requires them.

Calculate net expectancy in comparable units #

If gains average 1.8R, losses average 1R and costs average 0.12R, a 45% win fraction gives 0.45 × 1.8 − 0.55 × 1 − 0.12 = 0.14R per trade. This describes the specified hypothetical distribution; it does not estimate the true next-trade return. The break-even win fraction is (1 + 0.12)/(1.8 + 1) = 40%.

Use actual realised gains and losses for descriptive results, including partial exits, scratches and failed fills. Do not force every trade into a 2R/−1R pair when real execution did not produce those outcomes. Record the rupee value of R at entry and identify whether a multiple uses planned risk or realised loss.

Treat every trial as part of the research record #

Trying many parameter combinations and reporting only the best makes chance look like skill. Record failed variants, the number of configurations considered and why a candidate was selected. Changing the strategy after inspecting a holdout consumes that holdout as research information. It is no longer untouched evidence.

Use a chronological development period and a later evaluation period. For overlapping multi-bar trades, prevent training and evaluation observations from sharing information through the same underlying outcomes. Walk-forward testing must specify when recalibration occurs and which information is available at each step.

Inspect uncertainty and dependence #

A sample win fraction is not the true win probability. Even with 100 independent observations near 50%, the rough standard error is about five percentage points; dependence can reduce effective information further. Normal confidence approximations are especially fragile with small samples and extreme payoff tails.

Report trade counts, average and median returns, cost share, adverse excursions, losing runs, drawdown, exposure time and regime breakdown. A high Sharpe ratio from a short, heavily selected test is not enough. Primary overfitting research motivates checking selection bias; it does not validate a particular strategy in this library.

Make forward paper testing genuinely forward #

Freeze a version, select a test horizon in advance and log every eligible opportunity including no-trades and operational failures. Record bid/ask or a conservative execution approximation, not a convenient midpoint filled after the fact. If both stop and target are touched within one OHLC bar, use finer data or a conservative explicitly stated resolution rule.

Set review criteria before results arrive: which failure invalidates the research, which result is inconclusive and what evidence permits another test. “Stop when profitable” creates a biased sample. No fixed minimum trade count magically proves an edge across every market and regime.

Hypothetical worked example

How a gross edge disappears

Twenty fictional trades produce gross gains of ₹12,000 and gross losses of ₹9,000: gross net ₹3,000. If total execution, fees and slippage are ₹3,400, net P&L is −₹400. Reporting 60% winning trades without the cost ledger would miss the main result.

Keep a cost-stress column. Doubling an assumed ₹0.50 per-unit friction to ₹1.00 can change an apparently viable short-horizon method substantially. Recalculate the strategy from the ledger rather than subtracting an arbitrary percentage from a headline return.

Common mistakes

Tuning the rule after inspecting the holdout and calling the result out-of-sample.
Ignoring strategies or parameters that failed during selection.
Reporting profit without exposure, costs and drawdown.

Check your understanding

You changed a rule after seeing the holdout results. How should that period now be treated?

Make this chapter useful

Can you defend the reasoning?

Before moving on, answer the self-check in your own words and write why each alternative fails. Use the worked example to test that explanation. Then apply it in the module research task; keep the assumptions and the evidence that could change your answer.

Sources and further reading

Further-reading references from the curriculum source list. Contract sizes, expiry calendars, settlement references, margin parameters and tax treatment can change; verify the current operative documents for the actual product. Worked figures are illustrative, not current quotes.

Progress is stored only on this browser.

Back to Trading playbooks and professional process · Learning library · Calculators

General financial education. No securities recommendation, assured outcome or personal advice. Read the risk disclosures.