Meta-labeling, read closely

Suppose you have a rule that says “go long.” A moving-average crossover, a momentum score, a classifier's up-vote — it doesn't matter. The rule fires, you take the trade, and often enough it works that the rule looks like it has an edge. But “often enough” hides two very different failures. The rule can be too eager — firing on trades that were never going to pay — or too shy, sitting out winners. In the language of classification these are false positives and false negatives, and the two ratios that name them are the whole story of this post:

• Precision — of the trades you actually took, what fraction paid off. Low precision means you are bleeding on bad bets.
• Recall — of all the trades that would have paid, what fraction you took. Low recall means you are leaving money on the table.
• F1 — the harmonic mean of the two, a single number that punishes you for being lopsided.

Most naïve strategies live at one extreme: fire on everything, catch every winner (high recall) and drown in losers (low precision). Meta-labeling is a structured way to walk that trade-off back toward the middle without touching the original rule.

Joubert's framework, consolidating an idea from de Prado's Advances in Financial Machine Learning, decomposes a meta-labeled strategy into three parts. The clean separation is the point:

• 1. The primary model (the side). Whatever you already have. It decides direction — long or short — and, crucially, it is tuned for high recall: let it be trigger-happy, flag every plausible trade, and don't worry yet about the false positives.
• 2. The meta-model (the filter). A binary classifier trained only on the trades the primary flagged, whose label is simply did that trade actually pay off? Its output is an estimated probability that the primary's call is correct.
• 3. Position sizing. Turn that probability into a bet size — act only above a confidence threshold, or stake in proportion to it. (We focus on the filter; sizing is in §07.)

To see the mechanism cleanly we need a world where we know exactly why each trade won or lost. So we build one (seeded, reproducible). Each opportunity has two hidden numbers drawn independently from a standard bell curve: a signal the primary rule can see, and a regime feature it cannot. A long trade pays off when their weighted sum clears zero:

profitable = 1 if 1·signal + 1·regime + noise > 0

The primary rule is deliberately simple: go long whenever the signal exceeds -0.4 — a low bar, so it flags 65.6% of opportunities and catches most winners. But because profitability also rides on the regime it never sees, plenty of those flagged trades are doomed. On 8,000 held-out opportunities the primary's precision is only 63.6%: for every three trades it takes, roughly one is a loser it could not have known to skip. That is the false-positive problem meta-labeling exists to attack.

def precision_recall_f1(acted, profitable):
    tp = sum(a and y       for a, y in zip(acted, profitable))
    fp = sum(a and not y   for a, y in zip(acted, profitable))
    fn = sum((not a) and y for a, y in zip(acted, profitable))
    p  = tp / (tp + fp) if tp + fp else 0.0   # of the trades we took, how many paid?
    r  = tp / (tp + fn) if tp + fn else 0.0   # of the winners, how many did we take?
    f1 = 2 * p * r / (p + r) if p + r else 0.0
    return p, r, f1

def average_precision(scores, profitable):
    # Rank trades by score, then lower the threshold one trade at a time.
    order = sorted(range(len(scores)), key=lambda i: -scores[i])
    total = sum(profitable)
    tp = fp = ap = prev_r = 0
    for i in order:
        tp, fp = (tp + 1, fp) if profitable[i] else (tp, fp + 1)
        r = tp / total
        p = tp / (tp + fp)
        ap += (r - prev_r) * p        # area under the precision/recall curve
        prev_r = r
    return ap

Those are the only metrics we need, implemented from scratch (the site runs the TypeScript; the Python and C++ are line-for-line translations). averagePrecision deserves a second look: it ranks the trades by a score, walks the threshold down one trade at a time, and integrates precision over recall. It is the area under the precision/recall curve — one number for “how well does this score separate winners from losers, at every operating point at once.” That single number is how we will judge each meta-model.

Now the meta-labeling step. We collect the primary's flagged trades, label each with whether it paid off, and fit a logistic regression to predict that label. We fit two versions — one that sees only the primary's own signal, and one that also sees the orthogonal regime feature — precisely to find out where the value comes from.

# Step 1 — the primary rule flags long trades from ONE signal.
flagged = [i for i in range(n) if signal[i] > primary_cut]

# Step 2 — meta-labels: among the flagged trades, which ones actually paid off?
meta_y  = [profitable[i] for i in flagged]

# Step 3 — a secondary classifier estimates P(this call is correct).
meta_signal = fit_logistic([[signal[i]]           for i in flagged], meta_y)  # signal only
meta_regime = fit_logistic([[signal[i], regime[i]] for i in flagged], meta_y) # + regime

def act(model, i, feats, theta):
    return predict_proba(model, feats) >= theta   # take the trade if confident enough

# The catch: proba(meta_signal, [signal]) is a MONOTONE function of signal, so a
# threshold on it is exactly a higher primary cutoff. Only meta_regime, which reads a
# feature the primary never saw, can bend the precision/recall curve.

Each meta-model gives every flagged trade a score, its estimated probability of paying off. Ranking the trades by that score and sweeping the threshold traces a precision/recall curve. We compare three rankings: the primary's raw signal (equivalent to simply raising its cutoff — the free baseline), the signal-only meta-model, and the signal + regime meta-model.

The headline — “a second ML model raised precision and F1” — is true and also deeply misleading, because almost any way of being pickier raises precision. If you simply demand a bigger signal before trading, you take fewer, better trades and precision climbs. The question that matters is whether the meta-model adds information, or just re-discovers a stricter cutoff you could have set by hand.

Fig 1 answers it. The signal-only meta-model's probability, σ(w·signal + b), is a monotone function of the signal — bigger signal, bigger probability, always. So thresholding that probability sorts the trades in the identical order as thresholding the raw signal. The two curves don't just look similar; their average precision agrees to nine decimals (0.809 vs 0.809). A meta-model built on the primary's own features is an expensive way to move a threshold.

The regime model breaks free only because it reads a feature the primary never touched. Its fitted weights, [1.58, 1.71] on [signal, regime], put real weight on the second coordinate — it has learned that a flagged trade in a hostile regime is a false positive, something no amount of staring at the signal could reveal. That is the entire source of the lift.

Even with a genuinely informative meta-model, three cautions travel with the F1 number:

• Precision is bought with recall. Every filter trades one for the other. The regime model at 0.6 recall is skipping 40.0% of the winners the primary would have caught. Whether that is a better book depends on the payoff of the trades you drop versus the losses you dodge — F1 weights them equally, a real P&L does not.
• The meta-label needs the future. To know whether a flagged trade “paid off” you must define an outcome — a horizon, a profit target, a stop. De Prado's triple-barrier labeling does exactly this, and it is where lookahead leakage sneaks in if the barriers peek past the decision point.
• Calibration matters once you size. Filtering only needs the ranking, but the moment you stake in proportion to the probability, its level has to be trustworthy. Joubert reports that fixed sizing schemes benefit from probability calibration while data-driven ones largely do not — a distinction our filter-only demo quietly steps around.

• Swap the meta-model. Our filter is one logistic regression; nothing stops it being a decision tree or a random forest that catches non-linear “bad regime” pockets a line can't. The framework is agnostic — meta-labeling is a wiring diagram, not a specific classifier.
• Size, don't just filter. Replace the hard threshold with a bet proportional to the calibrated probability, and measure the Sharpe ratio, not F1 — the metric a trader actually spends.
• Stress the orthogonality. Slide the regime feature from fully independent toward fully redundant with the signal and watch the green curve collapse onto the grey. That sweep is the thesis of this post, drawn as a dial.

Meta-labeling is one of the more quietly useful ideas in applied quant ML, and also one of the most over-sold. It will not rescue a primary model with no edge, and it will not conjure precision from features the primary already exhausted. What it does — cleanly, and with a second model you can build in an afternoon — is let you spend new information on the narrow, tractable question of when to trust the model you already have. Go turn the knobs on the logistic-regression page and picture it sitting downstream of a noisier friend, quietly vetoing its worst ideas.