The censored-demand guarantee that borrows its own CDF

A sale is not always demand. If the shelf empties at ten units, true demand may have been twelve or thirty: the label is capped from above. If a flash promotion lifts sales to twenty, the retailer may instead want the lower, unpromoted baseline. Ye and colleagues treat those as right- and left-censored observations and train one quantile model on all three kinds of label.

The attractive headline is an end-to-end loss with a proper-scoring-rule guarantee. The detail worth reading twice is what that guarantee needs: the redistribution weights must be computed from the true conditional demand CDF, and censoring must be independent of true demand after conditioning on the features. In practice the true CDF is unavailable, so the current model supplies the weights that train its next version.

A quantile is a demand level with a chosen probability below it. The 80th percentile, for example, is exceeded in roughly one case out of five when a forecast is calibrated. Ordinary quantile regression minimizes pinball loss: it charges underprediction at rate τ and overprediction at rate 1 − τ.

A stockout gives a lower bound R on latent demand: y > R. A promotion is represented as an upper bound L on the desired baseline: y < L. Dropping both groups wastes information; treating R and L as exact targets bends the fitted distribution toward the operational limits.

For a right-censored label, the model estimates τR = F(R). If the target τ lies above τR, the weight on the observed boundary is (τ − τR)/(1 − τR); the remaining mass is placed at a sufficiently high anchor. Left censoring is the mirror image. An uncensored example keeps the ordinary pinball loss.

def redistribution_weight(kind, tau, censor_tau):
    if kind == "right":
        return (tau - censor_tau) / (1 - censor_tau) if censor_tau <= tau else 1
    return (censor_tau - tau) / censor_tau if tau <= censor_tau else 1

def pinball(y, prediction, tau):
    residual = y - prediction
    return tau * residual if residual >= 0 else (tau - 1) * residual

def dpe(kind, observed, prediction, tau, censor_tau, low, high):
    if kind == "none":
        return pinball(observed, prediction, tau)
    weight = redistribution_weight(kind, tau, censor_tau)
    anchor = high if kind == "right" else low
    return weight * pinball(observed, prediction, tau) + (1 - weight) * pinball(anchor, prediction, tau)

The site runs the TypeScript version. In a hand-verified right-censored example with τ = 0.8, F(R) = 0.2 gives weight 0.75. Holding the observed stockout fixed but changing the estimated F(R) to 0.7 lowers that weight to 0.33. The boundary did not change; the model's belief about it did. For piecewise-linear knots (2, 6, 10) at probabilities (0.1, 0.5, 0.9), the core q-search places an observed value of 8 at τ = 0.7.

The theorem says DPE is proper under independent censoring if the redistribution weights are correct. Correctness means τL and τR come from the true demand CDF. Algorithm 1 cannot access that CDF. It predicts a grid of quantile knots, stops gradients through those predictions, inverts them at each censoring boundary, recomputes the weights, and takes a gradient step on the weighted loss. This is a generalized EM procedure and the paper argues convergence to a stationary point under mild conditions.

Extended D-Calibration spreads a censored observation across probability bins rather than assigning it to the boundary bin. Across 500 synthetic runs its model-ranking correlation with oracle calibration is 0.740 by Kendall's τ and 0.842 by Spearman's ρ. Ordinary ECE is negatively correlated with the oracle ranking (−0.220 and −0.262): it penalizes probability beyond an observed stockout even though that mass may be exactly right.

On Maicai, DPE cuts Extended D-Calibration by 70.6% relative to excluding censored labels (0.05 versus 0.17), while sharpness widens from 9.82 to 10.30. On Youxuan, LogNormal is fractionally better calibrated than DPE (0.039 versus 0.040) but 41.3% less sharp. There is no single winner if the decision-maker has not said how calibration and concentration trade off.

The two retail datasets contain 8.52 million and 18.07 million records, with roughly 19% right censoring and 3.56–6.26% left censoring. They are proprietary. Public M5 and Favorita data lack the censoring indicators required by the method, so an outside reader cannot replay the 30-day training, one-day validation, and four-day test protocol on the same labels.

Promotion spikes are declared left-censored because the target is counterfactual baseline demand without promotion. That target is not merely hidden; it depends on a business definition of what demand would have been. The seven-day A/B test is encouraging, but the paper does not report confidence intervals, product counts, or warehouse-level clustering for the 8% overstock reduction.

Start with a small monotone quantile head on the site's Transformer page, then compare three audits: oracle weights on synthetic data, self-estimated weights, and deliberately perturbed weights. Stratify failures by censoring rate and by the gap between estimated and oracle F(L) or F(R). A linear quantile baseline is useful precisely because it makes misspecification visible.

Next, violate conditional independence on purpose: let stocking policy depend on a latent demand signal excluded from x. Measure whether Extended D-Calibration still ranks models correctly and whether the inventory policy remains Pareto superior. Finally, repeat the online experiment for multiple replenishment cycles with clustered uncertainty. The compelling result is operational; the next study should make its uncertainty operational too.