Boosting · /lightgbm

LightGBM-style boosting

Two LightGBM ideas on top of the shared boosting core: features are quantized into histogram bins (the grid on the canvas — splits can only land on bin edges), and trees grow leaf-wise, always splitting the highest-gain leaf next. Internals compares leaf-wise against level-wise on the same data.

class 0class 1filled = train, hollow = testhover to probe · click to pin

step 0BCE —train acc —test acc —

Data

datasetpoints150noise0.15seedsplitmouse

Additive model (shared boosting core)

F_{m} (x) = F_{m - 1} (x) + η f_{m} (x) = F

Histogram binning

split candidates per feature: n - 1 \to k - 1 scanned last split: c

Thresholds can only sit on bin edges — coarser but far cheaper than the exact scan.

Leaf-wise growth

next split = ar g leaf ℓ max Gain (ℓ) until L leaves

Level-wise splits every node at each depth; leaf-wise always takes the single best gain anywhere in the tree — compare both in Internals.

Leaf weight (same as XGBoost)

w^{*} = - \frac{G}{H + λ} = w^{*}

What's simplified here

Histogram binning uses quantile bins over the train split; real LightGBM bins by value distribution with far more engineering.
No GOSS (gradient-based one-side sampling) and no EFB (exclusive feature bundling) in v1 — both named here because they matter in the real thing.
The level-wise comparison ensemble gets the same rounds and leaf budget, with depth ⌊log₂ L⌋.