Combinatorial Purged Cross-Validation

Financial observations often overlap in time. A position opened today may depend on prices observed several days later, so a random split can allow the training sample to contain information connected to the test sample. Combinatorial purged cross-validation builds several train and test arrangements, then removes observations whose information intervals overlap.

The central construction

Suppose the ordered sample is divided into $N$ groups and each test set contains $k$ groups. The number of distinct train and test combinations is

{N \choose k} = \frac{N!}{k!(N-k)!}.

Here, $N$ is the number of ordered groups, $k$ is the number selected for testing, and $!$ denotes the factorial. The construction produces

\phi(N,k)=\frac{k}{N}{N \choose k}={N-1 \choose k-1}

complete test paths when the combinations are arranged so that every group appears equally often. The symbol $\phi(N,k)$ is the number of paths, not a performance score.

A compact view of three validation paths. Actual purging is determined by observation intervals, not merely by the visible distance between groups.

Purging and the embargo

Purging removes a training observation when its information interval intersects a test interval. An embargo adds a short exclusion period after a test group. These two operations address different forms of temporal connection: direct interval overlap and unusually close observations near a test boundary.

Step	What changes	Purpose
Grouping	Ordered observations are placed into $N$ groups	Preserve time order
Combination	$k$ groups form each test set	Create varied test conditions
Purging	Overlapping training observations are removed	Reduce direct information overlap
Embargo	Nearby observations after a test group are excluded	Reduce boundary dependence
Path assembly	Test predictions are joined in time order	Compare several complete histories

This procedure supplies a distribution of validation outcomes. It does not guarantee future profitability, remove every source of dependence, or repair a poorly stated prediction problem.

Validation paths after purging and embargo

The notebooks show both the construction of the splits and the results obtained from them. The first figure shows six of the fifteen train and test combinations used in the example.

Six combinatorial validation splits with training observations in blue, test observations in red, and removed observations in orange. — Blue observations are used for training, red observations for testing, and the narrow orange intervals are removed by purging or the embargo. The removals change with the location of each test set.

The next figure uses public BTC/USDT observations. It compares path accuracy for the standard, bagged, and adaptive versions of the procedure.

Accuracy for standard, bagged, and adaptive combinatorial validation across five paths in a public BTC and USDT example. — Accuracy changes across the five paths, while all three methods remain close to one half. The example illustrates why the variation across paths matters; it is not evidence of future trading performance.

Reading the results

The examples use RiskLabAI 3.2.0 to construct the splits. Each notebook shows the exclusions and path diagnostics directly, making it possible to compare the full set of validation paths instead of relying on one summary score.

Notebooks

All three notebooks are public and have passed full execution checks.


1. Enumerate valid group combinations — Split and path counts
2. Remove temporal overlap and assemble paths — Path-level diagnostics
3. Compare validation paths with a public data example — Figures and summary tables