Conformal prediction for portfolios, read closely

Every model on this site outputs a number: a predicted return, a class probability, a next token. But a point forecast is silent about its own uncertainty. “Tomorrow's return is +0.4%” and “tomorrow's return is +0.4%, give or take 5%” are wildly different bets, and only the second one tells you how much to stake. The usual fix — assume the errors are Gaussian and read a ±1.96σ interval off the normal table — is only as good as that assumption, and financial returns are famously not Gaussian: they have fat tails, and big moves cluster.

Conformal prediction (Vovk, Gammerman & Shafer, 2005) is a way to attach an honest interval to any predictor without assuming a distribution at all. You give it a trained model and a held-out chunk of data; it hands you back a radius. The promise: a fresh point lands inside forecast ± radius at least a fraction 1 − α of the time, where you pick α (the miscoverage rate — the slice you're willing to miss). No bell curve required. Kato's 2024 paper, Conformal Predictive Portfolio Selection, takes that idea to portfolio construction: forecast returns, wrap them in conformal intervals, and choose the portfolio from the intervals rather than the bare forecasts.

The version we implement is split (inductive) conformal prediction, and it is almost embarrassingly simple. Split your data three ways:

• Train. Fit any point predictor ŷ = μ̂(x). Ours is the humblest useful one — an AR(1) model, ŷₜ = c + φ·yₜ₋₁, which is just linear regression of a value on the one before it.
• Calibrate. On a second, untouched chunk, record how wrong the model was: the nonconformity scores sᵢ = |yᵢ − μ̂(xᵢ)|, one absolute residual per point.
• Predict. The radius q is a specific order statistic of those scores — the k-th smallest, with k = ⌈(n+1)(1−α)⌉. The interval is [ŷ − q, ŷ + q].

import math

def conformal_quantile(scores, alpha):
    # Lei et al. 2018: the k-th smallest nonconformity score,
    #   k = ceil((n + 1) * (1 - alpha)).
    n = len(scores)
    k = math.ceil((n + 1) * (1 - alpha))
    if k > n:                       # level unattainable from n points
        return math.inf             # -> unbounded interval
    return sorted(scores)[k - 1]

def interval(y_hat, q):
    return (y_hat - q, y_hat + q)   # symmetric band of radius q

That (n+1) and the ceiling aren't decoration — they are the finite-sample correction that turns a plausible heuristic into a theorem. If the calibration points and the test point are exchangeable (their joint distribution doesn't care about order — a weaker cousin of “independent and identically distributed”), then the true value lands in the interval with probability at least 1 − α, and at most 1 − α + 1/(n+1) (Lei et al., 2018). Distribution-free, finite-sample, for any model you plug in. The site runs the TypeScript above; the Python and C++ are line-for-line translations.

Claims of a guarantee deserve suspicion, so we test it. We simulate an AR(1) return process (seeded, so you get the same run every time), split it into train / calibrate / test, and measure how often the test returns actually land in the conformal band. We sweep the miscoverage α and average over 60 independent paths.

Coverage is a property of many predictions, so a rate alone is abstract. Here is one path: the realised returns over a test window, with the 90% conformal band drawn as its upper and lower edges. The band has a fixed width — split conformal with absolute-residual scores produces a constant radius 0.0344 — and 89% of the points in this window fall inside it, almost exactly the 90% promised.

Now the paper's move. If every asset's next return comes with an interval, you can select on the interval instead of the point. The most conservative rule takes the lower edge ŷ − q — the worst-case return the guarantee still (marginally) protects — and holds whichever asset's worst case is best. Compare three rules over a rolling backtest:

• Equal weight — hold everything, the honest baseline.
• Point forecast — hold the asset with the highest predicted return. Uncertainty-blind.
• CPPS (lower bound) — hold the asset with the highest conformal lower edge. Uncertainty-aware.

# One period of CPPS selection over K assets.
def pick(assets, history, n_calib, alpha):
    best_point, best_lower = -inf, -inf
    point_pick, cpps_pick = 0, 0
    for k, series in enumerate(history):          # each asset's realised path
        model = fit_ar1(series[:-n_calib])        # y_t = c + phi * y_{t-1}
        preds = [predict_ar1(model, series[t-1]) for t in calib_index]
        q = conformal_quantile([abs(series[t] - p)  # |residual| scores
                                for t, p in zip(calib_index, preds)], alpha)
        y_hat = predict_ar1(model, series[-1])
        lower = y_hat - q                          # worst-case of the interval
        if y_hat > best_point: best_point, point_pick = y_hat, k
        if lower  > best_lower: best_lower, cpps_pick = lower, k
    return point_pick, cpps_pick                   # mean-only vs interval-aware

The setup is deliberately stripped so the mechanism is unmistakable: all six assets have the same expected drift and differ only in volatility and how predictable they are. So there is no return to be had from picking a “better” asset — only risk to be avoided. (This is a seeded, illustrative toy, not the paper's data; it shows the shape of Kato's finding, not his numbers.)

The guarantee is genuine, but read the quantifier carefully. Coverage holds marginally — averaged over all the randomness — and only under exchangeability. Two cracks hide in that sentence, and the second is fatal in markets.

First, marginal is not conditional. A 90% band can cover 98% of the calm days and 60% of the turbulent ones and still average 90% — the guarantee says nothing about coverage on the days you care about. Second, and worse: exchangeability assumes the calibration data and the future are drawn from the same pool. Markets violate this by construction. Volatility regimes shift; a model calibrated in a placid stretch meets a storm. Watch what that does to the promise.

• Adaptive conformal (ACI). Gibbs & Candès (2021) let α drift online: miss too often and the band widens automatically. Dropping ACI in place of the fixed radius is the obvious repair for the Fig 4 collapse — and a clean before/after to measure.
• Normalised scores. Scale each residual by a local volatility estimate before taking the quantile. The band then breathes with the market — wider in the storm, tighter in the calm — trading the constant-width simplicity for conditional relevance.
• Size the bet, don't just pick it. We used the interval to choose one asset. The richer use is position sizing: stake in proportion to the lower bound, so confidence and direction jointly set exposure — closer to what a real book would do with this signal.

The forecaster inside every interval above is the plainest linear model there is — go turn the knobs on the linear-regression page and picture a calibrated band riding along with the fit. Conformal prediction is a beautiful, honest idea: a guarantee you can actually derive. Just remember it guaranteed the world stays exchangeable, and the one thing a market reliably does is refuse.