Structural Breaks

This chapter discusses methods for detecting structural breaks, which are transitions from one market regime to another (e.g., from mean-reversion to momentum). These breaks are valuable for ML strategies because they catch most market participants off guard, leading to irrational behavior (like holding losing positions) that creates profitable, high-risk/reward opportunities.

The chapter divides these detection methods into two main categories: CUSUM tests and Explosiveness tests.

CUSUM Tests

These tests detect a structural break by measuring if a cumulative sum of errors or deviations significantly departs from zero.

• Brown-Durbin-Evans CUSUM Test: This test uses recursive least squares (RLS) to get 1-step ahead recursive residuals ($\hat{\omega}_j$). A structural break is suspected if the cumulative sum of these standardized residuals ($S_t$) crosses a predefined threshold.

  • CUSUM Statistic:
    $$S_{t}=\sum_{j=k+1}^{t} \frac{\hat{\omega}_{j}}{\hat{\sigma}_{\omega}}$$

• Chu-Stinchcombe-White CUSUM Test: A simpler test that works directly on log-price levels ($y_t$) by assuming the null hypothesis is "no change." It measures the departure of the current price from a past reference price $y_n$.

  • Statistic:
    $$S_{n, t}=\left(y_{t}-y_{n}\right)\left(\hat{\sigma}_{t} \sqrt{t-n}\right)^{-1}$$
  • Critical Value: The test statistic is compared against a time-dependent critical value.
    $$c_{\alpha}[n, t]=\sqrt{b_{\alpha}+\log [t-n]}$$
  • To solve for the arbitrary start point $n$, the test is often run as $S_{t}=\sup _{n \in[1, t]}\left\{S_{n, t}\right\}$.

Explosiveness Tests (Bubble Detection)

These tests are designed to detect bubbles (exponential growth or collapse), which standard unit-root tests often miss. Standard tests are poor at distinguishing a stationary process from a periodically collapsing bubble.

• Chow-Type / Supremum Dickey-Fuller (SDFC):

  • Concept: Assumes the process switches once from a random walk ($\rho=1$) to an explosive process ($\rho>1$) at an unknown break date $\tau^*$.
  • Method: It fits an ADF-style regression with a dummy variable $D_{t}(\tau^{*})$ for the break.
  • Statistic: Since the break date is unknown, it takes the supremum (maximum) of the test statistic over all possible break dates.
    $$S D F C=\sup _{\tau^{*} \in\left(\tau_{0}, 1-\tau_{0}\right)}\left\{D F C_{\tau^{*}}\right\}$$
  • Flaw: It cannot detect multiple bubbles (e.g., a bubble-burst-bubble cycle).

• Supremum Augmented Dickey-Fuller (SADF):

  • Concept: This is the robust method for detecting multiple, periodically collapsing bubbles.
  • Method: Instead of one break date, it fits the ADF regression on a backwards-expanding window. For each end point $t$, it recursively tests all possible start points $t_0$. A spike in the SADF statistic indicates a bubble.
  • Statistic:
    $$S A D F_{t}=\sup _{t_{0} \in[1, t-\tau]}\left\{A D F_{t_{0}, t}\right\}=\sup _{t_{0} \in[1, t-\tau]}\left\{\frac{\hat{\beta}_{t_{0}, t}}{\hat{\sigma}_{\beta_{t_{0}, t}}}\right\}$$
  • Key Refinements:
    1. Use Log Prices: Always use log prices, not raw prices, as they provide a more stable model of bubble dynamics.
    2. Robustness: SADF is sensitive to outliers since it uses the sup (maximum). More robust alternatives include QADF (Quantile ADF, which takes the $q$-th percentile) and CADF (Conditional ADF, which takes the conditional mean of values above a quantile).

RiskLabAI Implementation

In our RiskLabAI library, we provide a robust implementation for the (G)SADF tests in the features.structural_breaks.structural_breaks module. The implementation is modular, breaking the complex problem into a series of clear steps.

1. Lagging: The lag_dataframe function creates a DataFrame with the necessary lagged features.
2. Matrix Preparation: prepare_data constructs the properly aligned dependent variable ($y$) and independent variable matrix ($X$) for the ADF regression, handling the specified constant type ('c', 'ct', etc.) and lags.
3. OLS Computation: compute_beta efficiently computes the OLS coefficients ($\hat{\beta}$) and variance-covariance matrix ($\hat{\sigma}_{\beta_{t_{0}, t}}$) for a given window of $y$ and $X$.
4. Test Statistics: We provide two main functions:
   • get_expanding_window_adf: Computes the standard ADF t-statistic over an expanding window, which is useful for plotting the test statistic's evolution.
   • get_bsadf_statistic: Computes the Backward Supremum ADF (BSADF) statistic by finding the supremum t-statistic across all possible expanding windows. This is the core test for detecting bubble origination.

• Sub- and Super-Martingale Tests (SMT):
  • Concept: An alternative to ADF that does not assume an autoregressive process.
  • Method: Tests for explosive trends (e.g., polynomial, exponential, or power) on expanding windows, similar to SADF.
  • Statistic: Includes a penalty term $\varphi$ to adjust for sample length.
    $$S M T_{t}=\sup _{t_{0} \in[1, t-\tau]}\left\{\frac{|\hat{\beta}_{t_{0}, t}|}{\hat{\sigma}_{\beta_{t_{0}, t}}\left(t-t_{0}\right)^{\varphi}}\right\}$$

Implementation: Drift-Burst Hypothesis (DBH)

To test the effectiveness of explosiveness and bubble detection algorithms (like SADF), we need synthetic data that exhibits these characteristics. In our RiskLabAI.data.synthetic_data.drift_burst_hypothesis module, we implement the Drift-Burst Hypothesis (DBH) model.

This model generates drift and volatility parameters for a bubble scenario, featuring a predictable "explosion" at the midpoint (t=0.5) of the series.

Methodology

The model defines drift and volatility as a function of time t (from 0 to 1), where the denominator approaches zero at the midpoint, causing a burst:

To prevent division by zero, the model uses a small explosion_filter_width to clamp the denominator near the explosion point.

API reference

RiskLabAI implements these in Python and Julia (signatures auto-generated from the package source):

def lag_dataframe(
    market_data: pd.DataFrame, lags: Union[int, list[int]]
) -> pd.DataFrame:

function lag_dataframe(data::AbstractVector{<:Real}, lags::Union{Integer,AbstractVector{<:Integer}})

def prepare_data(
    log_price_series: pd.Series, constant: str, lags: int
) -> tuple[pd.DataFrame, pd.DataFrame]:

function prepare_data(log_price::AbstractVector{<:Real}, constant::AbstractString, lags::Integer)

def compute_beta(
    y_window: np.ndarray, x_window: np.ndarray
) -> tuple[np.ndarray, np.ndarray]:

function compute_beta(y_window::AbstractVecOrMat{<:Real}, x_window::AbstractMatrix{<:Real})

def get_expanding_window_adf(
    log_price: pd.Series,
    min_sample_length: int,
    constant: str,
    lags: int,
) -> pd.Series:

function get_expanding_window_adf(
    log_price::AbstractVector{<:Real},
    min_sample_length::Integer,
    constant::AbstractString,
    lags::Integer,
)

def get_bsadf_statistic(
    log_price: pd.Series,  # <-- CHANGED: Accept Series
    min_sample_length: int,
    constant: str,
    lags: int,
) -> dict[str, Any]:

function get_bsadf_statistic(
    log_price::AbstractVector{<:Real},
    min_sample_length::Integer,
    constant::AbstractString,
    lags::Integer,
)

Full source: Python · Julia