Kalman Filter Hedge Ratio: Dynamic Regression for Pairs Trading

A fixed hedge ratio fits one coefficient to a formation sample and holds it constant. A dynamic hedge model instead treats the intercept and slope as latent quantities that can evolve through time. The Kalman filter estimates those quantities recursively, combining the previous estimate with the information in a new observation.

For two candidate assets, write \(y_t\) for the log price of one and \(x_t\) for the log price of the other. A changing long-run regression can be expressed as \(y_t=\alpha_t+\beta_t x_t+\epsilon_t\). The coefficient \(\beta_t\) describes the fitted relation at time \(t\); it is not automatically a share count, dollar-neutral position, or stable economic law.

This application uses the general prediction-and-update recursion of a linear state-space model. Kalman introduced the recursive filtering framework, and later econometric work applied it to regression coefficients that vary over time. A pair-trading use of a stochastic spread is one application, but its trading assumptions must be evaluated separately from the filtering algorithm.

Define the state vector and observation row as

\[ \theta_t= \begin{bmatrix}\alpha_t\\\beta_t\end{bmatrix}, \qquad H_t=\begin{bmatrix}1&x_t\end{bmatrix}. \]

The observation equation is \(y_t=H_t\theta_t+\epsilon_t\). It says that the observed log price is related to the latent intercept and hedge coefficient, plus observation noise. A common starting point lets both coefficients follow random walks:

\[ \theta_t=\theta_{t-1}+\eta_t,\qquad \eta_t\sim N(0,Q),\qquad \epsilon_t\sim N(0,R). \]

Here \(Q\) is the covariance of coefficient changes and \(R\) is the observation-noise variance. Setting \(Q=0\) makes the coefficients fixed in this model; positive \(Q\) allows them to drift. Other state transitions are possible, but each one makes a different claim about how the relationship behaves.

Using a diagonal \(Q\) adds the assumption that shocks to the intercept and slope are uncorrelated. Off-diagonal terms allow them to move together, but the sample must contain enough information to estimate that covariance. Any valid covariance matrix must be symmetric and positive semidefinite.

This simple specification assumes zero-mean process and observation disturbances that are serially uncorrelated and mutually independent. Those assumptions belong to the model; price observations do not automatically satisfy them.

At the end of \(t-1\), the filter has an estimate \(\hat\theta_{t-1|t-1}\) and covariance \(P_{t-1|t-1}\). The random-walk state equation predicts

\[ \hat\theta_{t|t-1}=\hat\theta_{t-1|t-1}, \qquad P_{t|t-1}=P_{t-1|t-1}+Q. \]

The estimate carries forward, while uncertainty increases by the allowed coefficient-change covariance. \(P\) describes uncertainty about the latent regression state under the specified model. It is not a forecast of the pair's trading loss or a complete measure of portfolio risk.

The one-step fitted observation is \(H_t\hat\theta_{t|t-1}\). Using the prior state matters because it was formed without the new \(y_t\). If a study instead calculates a residual after updating the coefficients with that same observation, the residual has already been pulled toward zero by the update.

Because \(H_t\) contains \(x_t\), the current explanatory price affects both \(S_t\) and the gain. When slope uncertainty is material, observations at different \(x_t\) values can carry different information about \(\beta_t\), although intercept-slope covariance also matters. Centering or rescaling \(x_t\) changes the intercept and coefficient units, so the corresponding elements of \(Q\) and the initial \(P\) must be transformed or re-estimated rather than copied unchanged.

When the new observation arrives, its one-step prediction error, or innovation, is

\[ v_t=y_t-H_t\hat\theta_{t|t-1}, \qquad S_t=H_tP_{t|t-1}H_t^{\mathsf T}+R. \]

The innovation variance \(S_t\) combines predicted state uncertainty and observation noise. The Kalman gain and state update are

\[ K_t=P_{t|t-1}H_t^{\mathsf T}S_t^{-1}, \qquad \hat\theta_{t|t}=\hat\theta_{t|t-1}+K_tv_t. \]

The gain allocates the new surprise between changing the estimated relation and treating the observation as noise. A larger gain means the filter adjusts more to this observation; it does not mean that the relationship is more reliable.

Suppose a hypothetical filter predicts \(\hat\alpha=2.0\) and \(\hat\beta=1.20\) when \(x_t=50\), giving \(\hat y=62.0\). If the observed \(y_t=62.8\), the innovation is \(0.8\). If the model's gain is \([0.08,0.006]^{\mathsf T}\), the updated intercept is \(2.064\) and slope is \(1.2048\). These invented values show the arithmetic only; they are not an empirical hedge ratio or evidence that the deviation will reverse.

The process-noise matrix \(Q\) says how much the regression coefficients are allowed to move from one observation to the next. A larger \(Q\) generally increases the gain and makes the estimate more responsive. It can follow a genuine change sooner, but it can also chase temporary noise and create rapidly changing hedge weights. A very small \(Q\) produces a smoother estimate that may lag a real break.

The observation-noise variance \(R\) describes variation in \(y_t\) that the relationship does not explain. Holding other quantities fixed, a larger \(R\) makes the filter trust each new observation less. Neither \(Q\) nor \(R\) is a universal tuning constant. Their scale depends on whether the model uses prices, log prices, normalized series, and the chosen sampling interval.

An analyst can estimate noise parameters by likelihood or prespecify them from a training window, but tuning them against the eventual trading period leaks outcome information. The starting state \(\hat\theta_{0|0}\) and its covariance \(P_{0|0}\) also matter: an uncertain initialization can create a period of rapid early adjustment. Report the initialization, parameter-estimation sample, and sensitivity to reasonable alternatives.

A filtered estimate \(\hat\theta_{t|t}\) uses observations through time \(t\). A smoothed estimate at \(t\) can use observations that arrive after \(t\). Smoothing can help describe historical latent states, but it is not available to a trader at the historical decision time.

The same timing issue affects estimation of \(Q\), \(R\), pair selection, and signal thresholds. If they are fitted once using the full sample, earlier estimates benefit from later observations even if the coefficient recursion itself is one-sided. For a real-time simulation, fit hyperparameters using only a formation period, freeze or update them on a declared walk-forward schedule, and make each position decision with information available before execution.

Observation time is not fill time. If the closing price at \(t\) is needed to compute \(v_t\), a backtest cannot assume a fill at that same close unless the execution process makes that feasible. Use the next executable quote or a defensible execution model and include spread, impact, fees, borrow, and financing.

For a pairs study, the pre-update innovation \(v_t\) is the surprise relative to the prior fitted relation. Dividing by \(\sqrt{S_t}\) gives a standardized innovation under the model. It helps compare surprises across periods with different estimated uncertainty, but it does not say how much either asset should be bought or sold, or whether a large surprise will reverse.

The post-update residual \(y_t-H_t\hat\theta_{t|t}\) is a different quantity because it uses the current observation to revise the coefficients. Mixing that residual with a pre-update threshold changes the meaning of a signal. Define the spread, timing, and threshold explicitly, and state whether the coefficient is prior, filtered, or smoothed.

Even when \(\beta_t\) is estimated from log prices, converting it into orders requires a separate position rule. Log-price weights do not directly specify share quantities or contract counts. Prices, contract multipliers, currencies, and account constraints determine dollar exposure; the intercept is not a traded asset. Rebalance costs and exposure drift matter between coefficient updates.

A slowly drifting coefficient may be a useful representation of gradual change, but it can also absorb a relationship that has broken down. A model with flexible states can keep the fitted residual visually small even when there is no stable long-run equilibrium. The filter does not test cointegration and does not guarantee that its residual is stationary.

Pair choice also matters. Screening many assets and keeping the pair with the smoothest in-sample residual selects on the same data used to claim stability. Corporate actions, stale prices, asynchronous closes, short-sale constraints, changing liquidity, and regime shifts can all alter estimated states or make a paper hedge difficult to execute.

Check residual behavior, state uncertainty, turnover, and coefficient paths alongside the fitted line. Use chronological holdout periods and compare against a fixed-ratio benchmark. A more adaptive estimate should earn its additional turnover and estimation risk after realistic costs; visual smoothness alone is not a performance result.

State which price or return series are modeled, how they are transformed, the observation frequency, sample dates, and which asset is on each side of the regression. Give the observation and state equations, transition matrix, \(Q\), \(R\), initialization, estimation method, and whether reported states are filtered or smoothed.

For a trading result, define the innovation or spread used for the signal, its normalization, decision timestamp, order delay, position sizing, rebalance rule, and all execution costs. Describe pair screening, training and test windows, how noise parameters are retuned, and how the strategy behaves after a break or missing observation. Compare fixed and dynamic coefficients on the same chronological splits.

Kalman's linear filtering paper develops the recursive framework. Hatanaka studies regressions with some time-varying coefficients. Elliott, van der Hoek, and Malcolm set out a stochastic model for pairs trading. For related topics, see the guides to the Kalman prediction and update, cointegration versus correlation in pairs trading, Johansen rank testing, and variance-ratio tests.