← Blog/blog/equity-transformer-history-horizon-instability

The 5-minute winner that lost the next year

More history sounds safer: more examples, less estimation noise. In markets, old examples can also teach relationships that no longer transfer. Chalvatzis holds one cross-sectional Transformer and one 572-feature library fixed, then changes only which months enter training. The answer is not “use more data.” It depends on how far ahead the prediction reaches.

The paper's sharper warning sits in the appendix. The retrospectively best individual configuration changes at all 8 forecast horizons between 2025 and 2026. A clean full-period winner table is therefore a map of this backtest, not a stable rule for the next one.

01

What the score measures

At each forecast origin, the model ranks stocks. Rank IC is the Spearman correlation between that ranking and the ranking of future returns: +1 means perfect ordering, 0 means no monotone relationship. The target is a future five-factor residual return—the part left after removing common market, size, value, profitability, and investment factor exposure. Scores are averaged within sessions and then across 451 sessions, so a reported 1.90% means correlation 0.019, not a 1.90% return.

02

Recent data win quickly, then run out of road

R6, R12, and R24 sample respectively 6, 12, or 24 months with a recency preference from widening historical pools. R6 reaches 1.90% Rank IC at five minutes, but falls to 0.32% at 120 minutes; R24 rises above it at the long end. The TypeScript core feeds the exact paper-reported rows to this chart.

R6: 6 monthsR12: 12 monthsR24: 24 months
Paper-reported full-period Rank IC for the three recency configurations. The horizontal positions are 5, 10, 15, 20, 25, 30, 60, and 120 minutes; connecting lines aid comparison rather than implying equal time spacing.
03

The ensemble wins three columns, not the argument

Averaging all twelve forecasts beats every individual point estimate at 15, 20, and 25 minutes. At the other five horizons it dilutes a stronger model. The calculation below is deliberately signed: positive means the ensemble wins; negative means the best individual wins.

def ensemble_gaps(rows, ensemble_name):
    ensemble = next(row for row in rows if row.name == ensemble_name)
    individuals = [row for row in rows if row.name != ensemble_name]
    gaps = []
    for horizon in range(len(ensemble.values)):
        best = max(row.values[horizon] for row in individuals)
        gaps.append(ensemble.values[horizon] - best)
    return gaps
HorizonFull bestBest IC %Ensemble %Ens. gap2025 best2026 best
5mR61.901.53-0.37R6U24
10mR61.571.45-0.12M6C6
15mR61.001.10+0.10C12U24
20mM121.261.31+0.05M6M12
25mM241.081.17+0.09U12M24
30mM241.090.98-0.11U12M24
60mU240.870.81-0.06U12U24
120mU120.760.63-0.13U12M24
Paper-reported Rank ICs with core-computed winners and ensemble gaps. Full best is selected retrospectively. Stable row order resolves exact printed-value ties, so tied 2025 columns should not be over-interpreted.
04

A full-period mean hides the regime break

The equal ensemble's 5-minute IC is 3.72% in the first half of 2025 and 1.01% in 2026 through 20 October: only 27% of the earlier score remains. Retention is below one at every horizon. That does not prove “alpha decay”— market conditions, stock coverage, labels, and sampling noise can all move the estimate—but it does defeat any claim of stable superiority.

Later-to-earlier retention: the paper's 2026 ensemble Rank IC divided by H1 2025 Rank IC. These are derived ratios of reported results, not new backtest observations.
05

Why the horizon story remains an association

Each policy changes more than “freshness.” Wider training quantities also widen the eligible historical pool; sampled policies change month composition and validation behavior. The data are proprietary intraday US equity records, the architecture is fixed, and transaction costs are not the evaluation target. Rank IC says whether the ordering contains signal, not whether latency, turnover, spreads, and market impact leave a tradeable strategy.

06

What to probe next

Pre-register a history-selection rule using only past months, then compare it with fixed R6, U24, and equal-ensemble baselines in a nested walk-forward test. Report paired monthly uncertainty, turnover, and net returns. Most importantly, treat history length as a time-varying policy choice whose own selection can overfit—not as another row to maximize on the final table.

The site's Transformer page exposes the attention computation used by the model family; the linearpage helps unpack rank correlation's covariance logic. This paper's lesson lives one level above architecture: what the model is allowed to remember.

References

  1. Chalvatzis, C. (2026). Which Past Matters? Training History and Multi-Forecast Horizon in Equity Transformers. arXiv:2611.23126 [q-fin.ST]
  2. Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems 30