Off-Policy Evaluation and Policy Selection

A historical record often comes from one decision rule while the analyst wants to assess another. These notebooks use recorded action probabilities, outcome models, and held-out evaluation to estimate the value of a proposed rule without pretending every action was equally likely.

The central relation

The inverse-probability estimate of policy value is

\widehat V_{\mathrm{IPW}}(\pi)=\frac{1}{n}\sum_{i=1}^{n} \frac{\pi(A_i\mid X_i)}{\mu(A_i\mid X_i)}Y_i.

$X_i$ is the observed context, $A_i$ is the action taken, and $Y_i$ is the resulting outcome. The proposed policy probability is $\pi(A_i\mid X_i)$ , while $\mu(A_i\mid X_i)$ is the probability used by the historical behavior rule. The sample size is $n$ .

Estimating policy value from existing decisions

The overlap experiment shows how rare actions create large weights and unstable estimates. The selection experiment then separates the data used to choose a policy from the data used to assess its value.

Policy value error and variability as overlap between policies changes. — Poor overlap increases the influence of a few observations and raises sampling variability. A value estimate becomes unreliable when the behavior rule rarely takes actions favored by the proposed rule.

Held-out values for candidate decision policies. — Candidate policies are compared on observations not used for selection. This separation reduces the optimism created by choosing and judging a policy on the same outcomes.

The public basket returns support a transparent numerical example. The causal interpretation still relies on recorded action probabilities, overlap, and the absence of unmeasured causes of both action and outcome.

Notebooks


1. Identifying policy value under overlap
2. Doubly Robust Evaluation
3. Chronological Policy Selection