The pricing cartel that deters only the wrong deviation

Two pricing agents learn together, stop exploring, and settle above the competitive price. Force one firm to undercut for a period and its rival cuts too; soon both prices recover. That movie looks like retaliation. Lopes de Moraes and De Losso ask whether the frozen lookup tables actually contain a threat—or whether a stable dynamical system merely falls back into its only attractor.

The market is a Bertrand duopoly: two firms choose prices simultaneously. Demand uses a logit choice model, so a lower price wins more customers without receiving the whole market. Each tabular Q-learning agent observes only last period's price pair. On the 15-price grid that makes 225 states, each with 15 possible actions.

Training and testing are deliberately separated. During the audit, Q-tables are frozen, exploration is off, and greedy policies define a deterministic map from one joint-price state to the next. This matters: continued learning could repair a bad rematch and would blur what the deployed policy already carries.

Any deterministic map on a finite state space eventually enters a cycle. The mechanism question is therefore not “did prices return?” but “were there other destinations?” In 89.3% of trained pairs, one attractor owns all 225 starting states. A forced price cut and a cooperative price increase must both return to the same place because the map gives them nowhere else to go.

Fixing the rival's greedy policy turns the problem into an ordinary discounted Markov decision process. For every state and candidate price, value iteration includes today's logit-Bertrand profit and the value of the next joint-price state. The maximizer is a complete contingent plan, not a one-period cut followed by surrender.

def best_response(prices, partner, discount, tol=1e-12):
    n = len(prices)
    values = [0.0] * (n * n)
    policy = [0] * (n * n)
    while True:
        next_values = []
        for state in range(n * n):
            rival_action = partner[state]
            candidates = []
            for own_action in range(n):
                next_state = own_action * n + rival_action
                reward = profit(prices[own_action], prices[rival_action])
                candidates.append(reward + discount * values[next_state])
            best_action = max(range(n), key=lambda a: candidates[a])
            policy[state] = best_action
            next_values.append(candidates[best_action])
        gap = max(abs(a - b) for a, b in zip(next_values, values))
        values = next_values
        if gap <= tol:
            return values, policy

This page runs the TypeScript implementation above. Python and C++ are line-for-line translations. The three-price example below is explicitly illustrative: it is a small frozen partner designed to expose the same logical gap, not a reconstruction of the authors' 2,048 learned Q-tables.

A punishment strategy should travel with the agent. The authors rematch frozen agents from independent training runs and average over all 225 initial states. The collusion index falls from 0.669 within the original pair to 0.134 with a stranger—an 80% collapse. Even rematching agents that converged to the same price does not rescue coordination.

This is an out-of-distribution test in unusually concrete clothing. Each policy was trained only on states generated by one particular partner. Away from that narrow path, the table is mostly untested. The result is a jointly carved basin that works at home and loses its object when the partner changes.

A stronger mechanism audit should freeze policies, enumerate basins, perturb both upward and downward, rematch agents without retraining, and solve the exact best response whenever the state space permits it. For larger function approximators, approximate planning and adversarial policy search can play the same role, but their optimization error must be reported.

The from-scratch neural-network explorer is a useful next stop for seeing how function approximation differs from a lookup table. The audit lesson survives the architecture change: behavioral resemblance is weaker evidence than a counterfactual test of the mechanism said to produce it.