Methods / research reasoning

The test matters only if you understand what it can and cannot tell you.

Methods explains research ideas independently from product controls. The goal is interpretation: what question a method addresses, what assumptions it makes, and what can go wrong.

Method anatomy / before interpretation
01 · QUESTIONWhat would change your conclusion?
02 · ASSUMPTIONSWhat world does the method presume?
03 · PROCEDUREWhat information enters each decision?
04 · LIMITSWhat remains unproven?
Core methods

Build a map of the evidence.

Use these methods to understand what each diagnostic measures, which assumptions matter, and where interpretation can fail.

Holdout evidence

In-sample vs out-of-sample

In-sample data helps develop or select an approach. Out-of-sample data is held back so evaluation is less entangled with the choices made during development. Repeated peeking at a holdout can weaken that separation.

See the visual concept →
Time-aware evaluation

Walk-forward analysis

Walk-forward analysis repeatedly moves a training/evaluation boundary through time. Its value comes from respecting temporal order and examining whether conclusions persist across multiple windows rather than one split.

See the visual concept →
Sequence uncertainty

Monte Carlo methods

Monte Carlo techniques generate many plausible resampled or simulated paths under stated assumptions. The useful output is a distribution of outcomes, not a guarantee that any particular path will occur.

Read the full method →
Decision and evidence structure

Validation foundations

Acceptance conditions, time-partition consistency, and evidence records answer different questions. Keep the rule, sample, and exact evidence identity attached to every validation label.

Read the validation foundations →
Neighborhood behavior

Parameter robustness

A result that exists only at a narrow parameter point can be more fragile than behavior that persists across a neighborhood. A surface view helps reveal plateaus, ridges, discontinuities, and isolated spikes.

See the visual concept →
Path-dependent risk

Drawdown

Drawdown measures decline from a prior peak, but one number can hide duration, clustering, recovery time, and repeated declines. Path context matters when interpreting it.

See the visual concept →
Cumulative path

Equity Curve

An equity curve traces a cumulative account or strategy value through an ordered sequence of observations or trades. Its shape can reveal path dependence, drawdowns, recovery periods, and concentration in time, but a smooth historical curve does not establish future performance. Interpretation depends on the valuation convention, costs, cash flows, leverage, and sampling interval used to build the series.

Shape and tails

Distribution shape

Mean and dispersion do not fully describe a distribution. Skew, tail mass, multimodality, and sample size can materially change interpretation, and estimated tails can be especially unstable in small samples.

Open the synthetic explorer →
Relationship structure

Correlation and diversification

Correlation describes co-movement under a chosen measure and sample. It can change across regimes, and low historical correlation does not prove future independence.

See the visual concept →
Multiple testing

Selection bias

When many candidates are tested, some can look exceptional by chance. Evaluation should account for the size and dependence structure of the search rather than treating the selected result as if it were the only trial.

See the selection funnel →
State definition

Market regimes

A regime groups observations by explicitly defined state variables or a documented model. Thresholds, feature choice, sampling, and persistence assumptions determine what the label means; hindsight labels are not predictions.

Open the synthetic regime cube →
Aggregate outcome

Net Profit

Net Profit sums the selected sample’s trade P/L under its accounting convention. The total does not describe path risk, capital efficiency, or how concentrated the outcome is.

Read the full metric reference →
Payoff balance

Profit Factor

Profit Factor compares gross gains with the absolute magnitude of gross losses over a defined sample. It can summarize payoff balance, but it does not describe trade order, drawdown duration, tail risk, or how concentrated the gains are. When gross losses are zero, the ratio needs special handling rather than casual interpretation.

Read the full metric reference →
Average trade outcome

Expectancy

Expectancy summarizes the average outcome per trade under a stated unit and cost convention. A positive historical average can still coexist with large dispersion, skew, regime dependence, or a small number of dominant trades. The unit, sample, and treatment of costs must be clear before values are compared.

Read the full metric reference →
Return versus path loss

Return / Drawdown Ratio

A return-to-drawdown ratio compares a chosen return measure with a chosen drawdown measure. Exact numerator, denominator, sign, and annualization conventions can vary by platform, so the ratio should not be interpreted without its definition. A high value can also reflect a short sample or unusually small observed drawdown.

Read the full metric reference →
Visual evidence

Reading Chart Evidence

Interpret OHLC bars, Volume Profile/TPO, and historical replay without treating chart geometry as a verdict.

Read the chart evidence guide →
Overfitting diagnostic

Probability of Backtest Overfitting

PBO-style diagnostics examine how often in-sample selection fails to carry into held-out partitions under a defined procedure. Interpretation depends on the partitioning method, candidate family, and selection statistic.

Selection-adjusted statistic

Deflated Sharpe Ratio

DSR adjusts interpretation of a Sharpe estimate for effects such as non-normality and multiple testing. It is a diagnostic, not a promise of future performance.

Choose the depth you need

Move from visual intuition to definitions, practical workflows, or product reference without losing the assumptions behind the analysis.