The gradient truce that depends on who speaks first

One model can learn several jobs at once: a street-scene network might label every pixel, estimate depth, and predict surface orientation from the same image. That is multi-task learning. Sharing a representation can save data and compute, but every job also pulls the shared parameters in its preferred direction.

A gradient is that pull: a vector pointing uphill in a task's loss. Training steps in the opposite direction. If two gradients have a negative dot product—or negative cosine similarity—the paper calls them conflicting. Improving one task locally can then hurt the other.

For gradient gᵢ and conflicting reference gⱼ, PCGrad computes gᵢ ← gᵢ − (gᵢ·gⱼ / ‖gⱼ‖²)gⱼ. The new vector is perpendicular to gⱼ, so its first-order effect no longer pushes that task uphill.

def project_conflicting(gradient, reference):
    inner = sum(g * r for g, r in zip(gradient, reference))
    squared = sum(r * r for r in reference)
    if inner >= 0 or squared == 0:
        return gradient[:]
    scale = inner / squared
    return [g - scale * r for g, r in zip(gradient, reference)]

This site runs the tested TypeScript implementation; Python and C++ are faithful translations of the same projection. No model internals are needed beyond one gradient per task.

Conflict alone is not the full diagnosis. The authors' “tragic triad” combines opposing directions, very different gradient magnitudes, and high positive curvature. In that setting a large task can dominate the sum, while curvature makes its apparent improvement look better—and the smaller task's damage look milder—than either really is.

With more than two tasks, PCGrad repeatedly projects one already modified gradient against the original gradients of the others. Vector projections generally do not commute: applying task 2 and then task 3 need not match applying task 3 and then task 2.

In the illustrative three-task set below, task 1 conflicts with both others; its cosine with task 2 is -0.707. The tested core enumerates all six global task orders. Nothing about the gradients changes, yet the final update does.

The standalone PCGrad network reached 71.0%, above independent training at 67.7%. Adding PCGrad to the stronger routing system moved accuracy from 74.7% to 77.5%. On NYUv2, PCGrad plus MTAN led eight of nine reported task metrics.

But Appendix J reports that the supervised runs took up to 12 hours and 10 GB on a TITAN RTX, versus 8 hours and 6 GB without PCGrad. On the MT50 reinforcement-learning benchmark, PCGrad used 5 days and 6 GB, versus 3 days and 3 GB for vanilla SAC. Separate task gradients and pairwise projections buy coordination with extra time and memory.

• Report variance across task-order seeds, not only the mean.
• Measure which tasks gain and lose, not just average performance.
• Compare wall-clock and memory at the same final quality.
• Check whether conflicts predict harm without the other two members of the tragic triad.

PCGrad changes the gradient before an ordinary optimizer updates a shared network. Explore that machinery on the neural-network page, or compare the simple vector geometry with the linear-model page.