The 20% update cost that changes with the denominator

A cloud forecaster does not need to relearn at every minute. Miao, Yang, and Cai monitor recent prediction error, compare it with a validation baseline, and take one gradient step only when the window looks worse. That is the useful idea in AdaptLSTM. The paper reports that it captures 54% of naive online learning's gain at 20% of the cost—but “cost” changes meaning across the experiment.

Concept drift means the relationship learned from past data no longer matches the live stream. AdaptLSTM first computes mean absolute error (MAE) on a held-out validation set. Online, it fills a queue with the latest errors and triggers when the queue mean exceeds alpha × validation mean. The queue is then cleared.

def detect(errors, reference, alpha, window):
    queue, triggers = [], []
    for index, error in enumerate(errors):
        queue.append(error)
        if len(queue) > window:
            queue.pop(0)
        if len(queue) < window:
            continue
        recent = sum(queue) / window
        if recent > alpha * reference:
            triggers.append(index)
            queue.clear()
    return triggers

The site runs the TypeScript implementation above. In the seed-free toy stream below, the validation mean is 0.10, the threshold is 0.12, and a three-point window first triggers at index 5. The lag is deliberate: a single spike cannot force an update.

On 301 high-variability machines from Alibaba's public 2018 trace, twelve minutes of history predict four future steps. The paper adds gradual, sudden, recurring, or mixed scale drift to the test split. Static means no online updates; naive online means one Adam step after every observation; AdaptLSTM updates only after a trigger.

Under sudden drift, AdaptLSTM closes 70.4% of the Static-to-Naive MAE gap. Across all four scenarios the paper reports 54%. The ordering also holds over ten seeds. None of that depends on the disputed denominator; the disagreement begins when the paper converts those gains into “per unit compute.”

The main ablation contains 42 trigger events over 301 steps. The update ratio ρ = 0.2 ranks the top 20% of nodes for attribution, but the gradient still uses all nodes. From the paper's own numbers, four different percentages can be computed without changing the run.

This also changes the headline efficiency ratio. Dividing 54% captured improvement by a 20% label gives 2.7×. Dividing by the measured 55.1% update-time share gives about 0.98× on the same normalization. Neither is universally “correct”: wall-clock update time, total end-to-end latency, energy, and FLOPs are different resources. The denominator must be named before the ratio can travel.

The detector is an accessible deployment pattern: validate the baseline, smooth live error, update once, and measure the trigger lag. It wraps an LSTM in the paper, but the same controller also wraps GRU and Transformer backbones. The core idea therefore cross-links naturally to the site's neural-network walkthrough and Transformer walkthrough.

The next experiment should make ρ operational: compute the gradient only on the selected nodes, then report realized FLOPs, end-to-end latency, energy, and accuracy. If shared-weight bias makes that fail—as the authors warn—then ρ is an attribution knob, not a compute knob. Separately, evaluate natural drift: the main Machine Trace experiments synthetically scale features, so trigger quality on launches, holidays, or cache invalidations remains an open deployment question.