The invariance the model never had to learn

A Raman spectrum is a one-dimensional chemical fingerprint: peaks mark molecular vibrations, while instruments and sample preparation add offsets, gains, baselines, and noise. A useful foundation model should preserve the chemistry when two laboratories process that fingerprint differently.

Wei, Wu, and Zou ask an unusually sharp question: when a frozen encoder transfers across preprocessing pipelines, did the encoder learn that invariance—or did a deterministic operation erase the difference before the first learned weight saw it?

Write the complete model as f(x) = g(R(x)): R normalizes each spectrum using its own location and spread, then the learned encoder g consumes the result. If a preprocessing transform T satisfies R(T(x)) = R(x), every possible encoder—trained, random, or deliberately useless—must return the same representation.

For center-and-scale normalization, equality holds exactly when one signal is a positive multiple of the other plus a constant. Gain and offset become invisible by algebra, not by representation learning.

def normalize(x):
    center = sum(x) / len(x)
    spread = (sum((v - center) ** 2 for v in x) / len(x)) ** 0.5
    if spread == 0:
        raise ValueError("constant signal")
    return [(v - center) / spread for v in x]

The live figures run the tested TypeScript core; Python is the default translation. The paper deploys a robust version using the median and a percentile spread, plus clipping and a small stabilizer. The simpler mean-and-spread form above has the same positive-affine identity and makes the mechanism transparent.

The evaluation trains a simple classifier, called a linear probe, on frozen features produced under one preprocessing pipeline and tests it under another. Balanced accuracy averages recall across classes, preventing a large class from dominating the score. Off-diagonal cells are the real transfer test because train and test preprocessing differ.

The core arithmetic recovers the paper's distinction: normalization transfers at 52.0%, degrades by 42.0 points, and retains 55.3%. The encoder transfers slightly less at 50.5%, yet its ratio is nominally a hair better at 55.6%. Robustness ratios and useful accuracy can point in different directions.

An earlier protocol made the model look 18 points better than raw spectra on retention. Two of twelve off-diagonal comparisons connected inputs that normalization already collapsed. Removing those cells changed the gap to −1.2 points; replacing the ratio with absolute accuracy left it at −1.3. Against normalization rather than raw input, the final out-of-family gap was −0.4.

The deployed robust normalization divides by spread plus 10⁻⁶ and clips to [−8, 8]. Those engineering guards break exact equality. In the paper, more than 99% of 3,291 spectra stayed below a residual of 10⁻³, but the maximum reached roughly 3×10⁻³. Calling the transformation “removed” without saying ideal or deployed would overstate the result.

• Compute R(T(x)) − R(x) before training and report its distribution, not only a pass/fail threshold.
• Place intended augmentations after normalization if the encoder is supposed to learn them.
• Compare trained features with normalization alone and random or width-matched controls.
• Separate transfer accuracy, degradation, and retention.
• Group repeated measurements from the same physical specimen inside one fold.
• Test cross-instrument and cross-dataset tasks where raw normalized channels cannot serve as a shared representation.

The lesson applies well beyond spectroscopy and directly to the Transformer page: audit every deterministic front end before crediting a learned back end. If an augmentation never reaches the first parameter, no amount of contrastive training can teach the model to ignore it—the software already did.