The fraud model that looked safe after deleting the attacks

A fraud detector sees rows, not pixels. Amount must stay positive; a card brand cannot be 40% Visa and 60% Mastercard; a borrower cannot edit a credit bureau's history. That makes an ordinary adversarial example—a small input change that flips a prediction—an incomplete test. The changed row also has to be a valid financial record that the attacker can actually create.

Zeng and colleagues make the evaluation protocol itself the experiment. They hold the split, model, attack budget, and seed fixed, then compare an unconstrained attack, filtering invalid outputs afterward, and generating attacks with constraints inside the loop. On Lending Club credit-risk data, the last choice changes the count of realistic successful attacks by nearly three orders of magnitude.

Protocol A ignores business rules and is a raw stress test. Protocol B runs that same attack and keeps only outputs that pass a feasibility checker. Protocol C changes the search itself: after each update it repairs categories and linked fields; C2 also zeros updates to features outside the attacker's control.

A one-hot category stores one choice as one 1 and the rest 0. Projection repairs a fractional category by keeping its largest entry. A mutability mask is a vector of zeros and ones: multiplying the update by it freezes bureau- or platform-controlled columns. Feasibility and capability are therefore separate axes.

def attack(x, step, origin, eps, mask, project, constrained):
    update = step(x)
    if constrained:
        update = [u * m for u, m in zip(update, mask)]
    candidate = [v + u for v, u in zip(x, update)]
    candidate = [max(o - eps, min(o + eps, v))
                 for o, v in zip(origin, candidate)]
    if constrained:
        candidate = project(candidate)
    return candidate

The TypeScript used by the figures is the tested core implementation; Python is the default translation. The important ordering is update, capability mask, geometric budget, then domain projection. Filtering after the return statement cannot redirect the search toward a valid alternative.

Consider 24 synthetic positive-class rows with two numeric fields and a two-way category. A one-step sign attack moves every coordinate against a from-scratch linear score. Below a category-switch threshold, the unconstrained row becomes fractional and Protocol B discards it. Projection can land directly on the other valid category; masking then tests whether the remaining editable fields still cross the boundary.

On IEEE-CIS, projection raised the paper's feasible flips from 0.0 to 93.7, but adding the capability mask cut them to 4.3: much of the useful attack direction touched protected features. Lending Club moved the opposite way, from 2,071.0 to 2,832.3. The surprising increase is a reminder that projection, masking, and optimization interact; a mask can stop updates that were steering the search away from a successful valid point.

That is why “constraint-aware” cannot be one checkbox. A valid row may still be impossible for this attacker, and an editable row may violate a cross-field contract. Report both.

FraudBench is admirably explicit that ε = 0.1 in standardized feature space is a first-order stress test. Standardization makes unlike columns numerically comparable, but it does not price effort. Moving income by half a standard deviation, changing a purpose field, and building six months of account history can have similar geometric size and radically different real cost.

• Publish the raw-to-processed mutability map with every benchmark.
• Report clean and robust PR-AUC beside feasibility and feasible flips.
• Replace standardized distance with money, time, detection risk, or expected profit.
• Run more seeds when the positive class is below one percent.
• Use decision-based attacks for trees instead of treating a zero gradient as robustness.
• Test whether rankings survive alternative valid constraint catalogues.

The practical lesson crosses both the logistic and XGBoost pages: robustness is a property of a model, a threat model, and an evaluator together. A detector does not become safer because the evaluation searched the wrong space and then threw its mistakes away.