Integrated Gradients explains the difference from whatever you call nothing

A neural network says an image is a fireboat. Which pixels deserve the credit? A feature attribution assigns a number to each input feature: positive values support the score, negative values pull against it. The tempting answer is the gradient at the image—the slope of the score when each pixel moves a tiny amount.

That slope can lie by omission. A sigmoid or ReLU network can be flat at the observed input even when moving from an empty input to the real one changed the prediction completely. Integrated Gradients fixes this by collecting slopes all the way from a baseline to the input. The baseline is a counterfactual: the input that is supposed to mean “the feature is absent.”

Start at the baseline, move in a straight line toward the input, and evaluate the model's input gradient along the route. Average those gradients, then multiply feature by feature by the total displacement. In calculus notation, feature i receives (xᵢ − x′ᵢ) ∫₀¹ ∂F(x′ + α(x − x′))/∂xᵢ dα.

def integrated_gradients(x, baseline, gradient, steps):
    delta = [x[i] - baseline[i] for i in range(len(x))]
    totals = [0.0 for _ in x]
    for step in range(steps):
        alpha = (step + 0.5) / steps
        point = [baseline[i] + alpha * delta[i] for i in range(len(x))]
        sample = gradient(point)
        totals = [totals[i] + sample[i] for i in range(len(x))]
    return [delta[i] * totals[i] / steps for i in range(len(x))]

The implementation uses midpoint samples for a slightly cleaner numerical approximation than endpoint samples. It needs only a gradient function—no access to layer internals and no ML library. The paper reports that 20 to 300 gradient calls usually approximated the integral within 5%, and recommends checking the sum before trusting the explanation.

For a differentiable model, the exact attributions sum to the score difference between the endpoints. That is completeness. It catches a coarse numerical integral because the sum will miss its target, but it cannot tell you whether the baseline is meaningful.

Consider the tiny model F(x₁,x₂)=x₁x₂ at input (1,1). From baseline (0,0), both features grow together, so symmetry gives each half the score difference. From baseline (0,1), the second feature is already present; only the first changes, so it receives everything.

The first explanation sums to 1.00 with residual 0.00. The second also sums to 1.00 with residual 0.00. Both pass completeness exactly; they answer different counterfactual questions.

The paper applies the method to two image models, two language models, and a chemistry model. The demonstrations include diagnosing a molecular network architecture and surfacing image regions used by a diabetic-retinopathy model. These are qualitative applications, not a universal accuracy benchmark for explanations—the authors stress that empirical attribution evaluation confounds model, data, and explainer artifacts.

• Run several defensible baselines and report attribution spread.
• Check completeness at increasing step counts before interpreting pixels.
• Test whether interpolated path inputs remain plausible to the model.
• Separate single-feature credit from feature-interaction analysis.

The gradient comes from ordinary backpropagation; poke at it on the neural-network page. For a linear model, the path integral collapses to coefficient times displacement, which you can compare on the linear-model page.