The 10-sigma detector that fires on 36% of noise

Suppose an anomaly detector selects the rows in your dataset. Its labels are not a fixed property of the detector: they also depend on how common real anomalies are where you deploy it. Huang, Wan, and Ou show the consequence in a financial event study built from scheduled US macroeconomic announcements.

Their detector was meant to find missing announcement timestamps from sharp two-minute price moves. One line divided each two-minute move by a one-second volatility estimate without multiplying the denominator by √120. That unit mismatch is the post's spine: a rule described as 10 sigma was only 10/√120, or 0.91 sigma, under the paper's Gaussian diagnostic.

Prevalence is the fraction of candidate windows containing a real event before the detector runs. Precision is the fraction of detector hits that are real afterward. Bayes' rule connects them: precision equals prevalence × true-positive rate, divided by that quantity plus (1 − prevalence) × false-positive rate.

This matters whenever model outputs define a dataset: fraud alerts, medical screens, content moderation queues, weak labels, and anomaly windows all inherit the deployment pool's base rate. It is the same calibration lesson developed interactively on the logistic-regression page, now applied to selection rather than prediction alone.

def detector_fpr(threshold, steps):
    effective_z = threshold / sqrt(steps)
    return 2 * (1 - normal_cdf(effective_z))

def precision(prevalence, tpr, fpr):
    true_hits = prevalence * tpr
    false_hits = (1 - prevalence) * fpr
    return true_hits / (true_hits + false_hits)

The Python tab is the default; TypeScript and C++ are faithful line-for-line translations. The charts below call the tested TypeScript functions implementing the same equations.

Under an independent Gaussian null, the standard deviation of a sum of 120 one-second changes grows as √120. Dividing the two-minute change by a one-second standard deviation therefore inflates the z-score by √120. The nominal threshold 10 becomes an effective threshold of 0.913.

The Gaussian calculation is diagnostic rather than a faithful market model—high-frequency returns are neither perfectly independent nor Gaussian. That does not rescue the unit mismatch. It explains why a supposedly extraordinary cutoff could accept ordinary quiet-window moves so frequently.

Scheduled FOMC and payroll announcements are relatively sparse and well catalogued. CPI releases occur every month, but the legacy source stopped listing many timestamps after a database transition. The same detector was applied to pools with different true-event prevalence, so its precision could not transfer unchanged.

The paper's naive transfer predicted CPI precision of 0.955. Manual audit measured 0.183—a 422% relative overstatement. Plugging the CPI prevalence into Bayes' rule predicted 0.177, only 3.3% below the audit. The key correction did not require a more complicated classifier; it required measuring the pool.

The paper next screens the CPI phantoms and keeps 473 suitable observations beside 308 true releases, for a count contamination share c = 0.606. True events have mean surprise intensity 6.38 basis points; phantoms have 3.76, so their relative intensity ρ is 0.590. A basis point is one hundredth of a percentage point.

Because the event-study curves are intensity weighted, the effective phantom weight is cρ/(1−c+cρ), not simply c. Here it is 47.5%. The observed contaminated curve equals that weighted mixture of the true and phantom response curves to numerical residual 1.1 × 10⁻¹⁶ in the paper. Contamination carries structure.

With a denominator-free response, the paper's true releases have larger average moves than legacy detections, which in turn exceed phantoms. Divide by each event's signed surprise, however, and the legacy sample appears stronger than the official sample at both horizons. The estimator, not the market, changes the ordering.

The per-event ratio is fragile because surprises are signed and can approach zero. On 335 matched observations, the paper reports a mean of ratios of 0.3997 versus 0.1020 for the pooled ratio of signed responses to absolute surprise. Under a continuous denominator with density near zero, the former need not even have a finite expectation.

The paper's evidence is deliberately narrow: one futures instrument, one detector family, and a sample effectively beginning in 2013. Its selection-conditioned Gaussian null reproduces the phantom curve's shape but not its level (reported RMSE 0.092 against double-precision differences around 0.02–0.07). Those limitations make replication on other assets and detector thresholds the important next experiment.

The broader lesson is durable. Treat a detector as a measurement process, not a label oracle: align its units, retain the candidate-pool prevalence, audit each deployment stratum, plot denominator sensitivity, and carry uncertainty through the downstream model. If the later analysis is a linear response estimate, the mechanics on the linear-regression page are a useful next stop.