The neural net that puts a curve on every edge—and runs 10× slower

A normal neural network sends a number across each connection: multiply by a weight, add the results, then apply the same activation function at a neuron. A Kolmogorov–Arnold Network, or KAN, sends the number through a different learnable curve on every connection. The receiving neuron only adds.

That swap sounds cosmetic. It is not. A single KAN edge can bend into a square, sine, logarithm, or other one-variable shape. The network can expose those curves to a scientist instead of hiding everything inside a matrix of scalar weights.

A spline is a curve assembled from low-degree polynomial pieces. Its grid points are called knots. Changing one coefficient changes the curve locally, which makes splines accurate and inspectable for one-dimensional relationships.

The paper uses cubic B-splines plus a residual SiLU path. Our reconstruction uses the simplest degree-one spline so every operation is visible: locate the two surrounding knots and interpolate their learned coefficients. The TypeScript version below is the logic used by the live study; Python and C++ are faithful translations.

def linear_spline(x, knots, coeffs):
    if x <= knots[0]:
        return coeffs[0]
    if x >= knots[-1]:
        return coeffs[-1]
    i = 0
    while x > knots[i + 1]:
        i += 1
    t = (x - knots[i]) / (knots[i + 1] - knots[i])
    return (1 - t) * coeffs[i] + t * coeffs[i + 1]

For output node j, a KAN layer computes x′ⱼ = Σᵢ φⱼᵢ(xᵢ). Each φⱼᵢ is its own spline. Compare that with an MLP's x′ = activation(Wx + b): the MLP learns the numbers in W, while the KAN learns the shapes on the edges.

The paper's clean toy is f(x,y) = exp(sin(πx) + y²). Its structure is exactly what a small KAN wants: learn sin(πx) and y² on two incoming edges, add them at one hidden node, then learn exp(z) on the final edge.

To isolate representation from optimization, this reconstruction gives the model that decomposition and fits each one-dimensional spline by least squares. It is oracle-decomposed, deterministic, and illustrative; it does not claim that gradient descent discovered the structure unaided.

An MLP connection stores one weight. A cubic KAN connection with G grid intervals stores roughly G + 3 spline coefficients. For equal layer widths, the paper counts KAN parameters as O(N²LG), versus O(N²L) for an MLP. Refining the curve therefore charges every edge.

Smaller KAN widths can still win the total trade-off, as the PDE example shows. But “fewer parameters” and “less compute” are not synonyms. Grid resolution, edge count, implementation, and the minimum width needed for the task all matter.

The Kolmogorov–Arnold representation theorem says a continuous multivariable function on a bounded domain can be composed from one-variable functions and addition. The awkward part is that those one-variable functions may be non-smooth or even fractal—terrible targets for a modest spline grid.

The paper's fast scaling argument therefore assumes a smooth KAN representation exists. Its experiments establish the favorable scaling on synthetic functions chosen to have that structure. The authors explicitly leave open whether comparable representations exist, and can be found by training, for tasks such as language modeling.

• Compare wall-clock time and energy at matched test error, not only parameter count.
• Hide the toy decomposition and measure how often training recovers the three true edge functions across seeds.
• Stress the grid with discontinuities, noise, and inputs outside the training range, where local spline accuracy is least reassuring.

To compare the ordinary weight-and-activation design directly, poke at the site's neural-network page. The familiar MLP is not as inspectable edge by edge, but its dense matrix operations are exactly why it trains so efficiently.