A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention
*Work done at Meta, †Corresponding Author
How to effectively train MLIPs at scale?
Our Answer: All-to-all Node Attention
AllScAIP pairs local neighborhood attention with global all-to-all node attention: every atom attends to every other atom in a single layer. Both stages use standard multi-head self-attention CUDA kernels ⚡⚡⚡
What does node attention actually learn? Click an atom to see: attention reaches far beyond the local cutoff.
Does it matter? Stretch a molecule and watch how the error changes.
What happens to inductive biases when data and model size scale up?
AllScAIP encodes physics at three levels: Enforced symmetries that never break, Optional geometric priors that help at small scale, and biases left entirely Learnable.
The Evidence: Component Ablations
Results
Open Molecules 2025
AllScAIP is on top of the OMol25 leaderboard at the time of release: see the leaderboard live!
Molecular Dynamics Simulations
NPT simulations recover experimental density and heat of vaporization out of the box: no more over-compression!
