The accurate yield model that lost the farmer money

A winter-wheat farmer chooses nitrogen before knowing the season's weather or final prices. More nitrogen can raise yield, but every extra kilogram costs money. The useful prediction is therefore not “how much grain?” It is “where does one more kilogram of nitrogen stop paying for itself?”

Wang and Yang test that distinction on 892 measured yield-response curves from two long-running UK field experiments. Their six machine-learning families include ridge regression, random forests, Gaussian processes, and TabPFN. The surprising result is not that a fancier model wins. It is that prediction accuracy barely ranks the decisions at all.

Let Y(N) be tonnes of grain per hectare after applying N kilograms of nitrogen per hectare. The paper compresses both market prices into b: kilograms of grain needed to pay for one kilogram of nitrogen. Profit in grain terms is π(N) = Y(N) − bN/1000. Profit loss, or decision regret, is the best attainable profit minus the profit from the recommended rate.

def profit(curve, nitrogen, price_ratio):
    intercept, slope, curvature = curve
    yield_t = intercept + slope * nitrogen - curvature * nitrogen ** 2
    return yield_t - price_ratio / 1000 * nitrogen

def optimal_nitrogen(curve, price_ratio, maximum=300):
    _, slope, curvature = curve
    unconstrained = (slope - price_ratio / 1000) / (2 * curvature)
    return max(0, min(maximum, unconstrained))

The quadratic is an explanatory model, not the paper's fitted field curve. At 100 kg/ha and b = 5, the tested core computes 6.1 t/ha in grain-equivalent profit. Differentiation gives the optimum where the yield slope equals b/1000.

The standard score asks whether advice lands within a tolerance such as 20 kg N/ha. But each measured curve rests on only a few plot yields. Leaving one observation out moved the curve's own estimated optimum by a median 23 kg/ha—more than the pass tolerance. Even a perfect predictor of the noisy benchmark would fail.

Profit also stays flat near the optimum. In the paper, a 20 kg rate error cost a median 0.023 t/ha in grain terms at b = 5, while rare misses could cost hundreds of pounds per hectare. Absolute rate error punishes harmless disagreement yet gives extreme economic misses no special weight.

ML alone doubled median loss relative to RB209, the UK's standard nutrient guide. The useful design begins with that strong baseline and adds a damped ML correction. Damping pulls model advice partway toward its field-group average; an uncertainty cap limits corrections where six models disagree. The model still contributes field-to-field variation—damping it all the way to the mean made results 15% worse than the plain model.

The hybrid's median improvement—0.035 to 0.033 t/ha—is tiny. The more credible benefits are elsewhere: bias falls from +6 to +1 kg/ha, the worst decile improves, and large errors fall from 42 to 37. This is closer to insurance than an average-profit jackpot.

Broadbalk supplied training and tuning data. Woburn, with different soil and rotations, was held back entirely. Applied there unchanged, the frozen correction cut median profit loss from 0.074 to 0.042 t/ha—a 43% reduction with Wilcoxon p = 0.034. Scrambling the field-group labels erased the gain, returning loss to 0.074.

• Train directly against held-out profit regret, then compare it with post-hoc correction under the same split.
• Use multiple training sites before claiming nationwide transfer.
• Report tails, bias, and uncertainty alongside the median.
• Stress-test price, weather, grain-protein premiums, storage, and application costs.
• Replace default emissions factors with field measurements before making carbon claims.
• Pre-register the correction family so a long improvement ledger does not quietly become model selection.

This is the practical bridge from the linear regression, random forest, and Transformer pages to real decisions: predictive models estimate a response surface, but a cost-sensitive rule turns that surface into action. Evaluate the action in its own units—or accuracy can improve while the decision gets worse.