A protein–ligand pose is one proposed 3D arrangement of a small molecule inside a protein pocket. A cofolding model can generate many plausible poses; the practical problem is choosing which one to inspect, simulate, or synthesize first.
Pyrite turns that problem into 1,500 pairwise choices. Its best hybrid agent asks an LLM to invent features, then gives those features to logistic regression. It reaches 74% accuracy on the curated post-AF3 split, ahead of expert chemists. The buried question is whether winning pairs produces the ordered candidate list a discovery pipeline needs.
What Pyrite actually measures
Each example shows two poses from the same protein–ligand query and asks which lies closer to the known crystal structure. The paper balances the binary task and counts abstentions as wrong. This makes per-pair accuracy clean and comparable, but it does not test whether all candidates can be combined into a coherent total order.
| Method | Post-AF3 curated | Post-AF3 all |
|---|---|---|
| Experts, decisive only | 57% | — |
| Vanilla logistic regression | 64% | 60% |
| Best direct reasoner | 67% | 67% |
| Best LLM featurizer | 74% | 71% |
How pairwise votes become a list
A minimal aggregator gives each pose one point per predicted pairwise win, then sorts by points. This is a Copeland-style ranking. The tested TypeScript below drives the diagnostic; Python and C++ are faithful translations.
def pairwise_accuracy(comparisons):
if not comparisons:
raise ValueError('comparisons must not be empty')
correct = sum(predicted == truth for predicted, truth in comparisons)
return correct / len(comparisons)
def rank_by_pairwise_wins(candidate_count, pairs):
scores = [0] * candidate_count
for left, right, winner in pairs:
if winner not in (left, right):
raise ValueError('winner must belong to pair')
scores[winner] += 1
return sorted(range(candidate_count), key=lambda i: (-scores[i], i))| One wrong pair | Pair accuracy | Errors touching best | Ranked first | Top-1 recall |
|---|---|---|---|---|
| Error: best vs second | 83.3% | 1 | Pose 2 | 0% |
| Error: third vs fourth | 83.3% | 0 | Pose 1 | 100% |
The benchmark itself changes the heuristic
Pyrite keeps the top 25 poses selected by MM-GBSA before constructing difficult pairs. Inside that already-filtered set, choosing the lower MM-GBSA energy is anti-correlated with the answer: 35% accuracy on the full post-AF3 split. The paper explicitly warns that flipping the sign is not a general physical law—the relationship changes with how many candidates survive the initial filter.
What the 74% does not yet establish
Training uses pre-AF3 structures and evaluation uses post-AF3 structures, so calendar is a useful leakage guard but also a possible distribution shift. The candidates come from one proprietary cofolding model and diversity sampler. Most importantly, the paper evaluates pairs, not the end-to-end top-k enrichment, regret, or cost of a full ranking pipeline.
The next experiment should group every candidate set by complex, score all pairs without reusing them across folds, aggregate them into a list, and report top-1/top-k recovery plus cycle rates. Repeat that protocol on poses from several public generators. The site's logistic regression page explains the auditable downstream model; decision trees provide the nonlinear baseline the paper found statistically tied with it.
References
- Srivastava, M., Lim, M. H., Keys, A. M., Murugan, P., and Gruver, N. (2026). Pyrite: Building and Benchmarking Protein-Ligand Pose Ranking Agents. NeurIPS 2026 Workshop on Agentic AI for Biological Discovery; arXiv:2611.14511
- Cox, D. R. (1958). The Regression Analysis of Binary Sequences. Journal of the Royal Statistical Society: Series B 20(2)
- Copeland, A. H. (1951). A Reasonable Social Welfare Function. University of Michigan Seminar on Applications of Mathematics to the Social Sciences