The loss that silences easy examples—and trusts every hard one

An object detector does not inspect one neat crop. It lays boxes across positions, sizes, and shapes, then asks whether each box contains an object. The paper calls these candidate boxes anchors. A one-stage detector evaluates roughly 100,000 of them per image, while only a few overlap real objects.

That creates extreme class imbalance: background examples outnumber foreground objects by as much as 1,000 to one. Even when each easy background box has a tiny loss, their sum can drown out the examples that still teach the model something. Focal loss changes who gets heard without throwing examples away.

Let p_t be the probability assigned to the correct class. Ordinary cross entropy is minus log(p_t). Focal loss multiplies it by (1 − p_t) raised to gamma. Gamma controls the focus: zero recovers cross entropy; larger values mute confident examples faster. Alpha separately balances positive and negative classes.

def focal_loss(logit, target, gamma=2.0, alpha=0.25):
    probability = 1.0 / (1.0 + math.exp(-logit))
    probability_correct = probability if target == 1 else 1.0 - probability
    alpha_target = alpha if target == 1 else 1.0 - alpha
    cross_entropy = max(logit, 0.0) - logit * target + math.log1p(math.exp(-abs(logit)))
    return alpha_target * (1.0 - probability_correct) ** gamma * cross_entropy

The stable cross-entropy expression works from the raw score, or logit, rather than taking the logarithm of a rounded probability. The tested TypeScript core also implements the exact gradient; the three snippets are faithful translations of the loss.

At 90% confidence, gamma 2 multiplies cross entropy by 0.1 squared: exactly a 100-fold discount. A wrong prediction is not erased. At 50% confidence its loss is only four times smaller, so the model spends a larger share of its update on uncertain and incorrect anchors.

Cross entropy assigns the easy crowd about 10.05 units of total loss, versus 5.11 for the ten harder examples. Focal loss flips the allocation: about 0.001 versus 0.817. It behaves like soft hard-example mining, but remains differentiable and uses every anchor.

The authors tuned alpha for each gamma while holding the detector and evaluation setup fixed. Performance rose through gamma 2, then fell at gamma 5. The mechanism was robust across a range, but “focus harder” was not a monotonic recipe.

• Inject controlled label noise and measure which examples dominate the gradient.
• Plot performance separately for rare, small, and ambiguous objects.
• Retune gamma when the foreground-background ratio changes.

Focal loss sits directly on top of binary classification. Poke at the probability and cross-entropy mechanics on the logistic-regression page, then follow the gradients through the neural-network page.