Summary
The announced run uses 80% pretraining and 20% midtraining across an 18.75T-token mixture over roughly three months, with estimated compute of 2.7e24 FLOPs. Marin says its data mixture, sampled documents and live training loss will be available through GitHub and Weights & Biases.
Why it matters
Publicly exposing a training run at this scale makes data composition and scaling behaviour more inspectable than typical closed-lab releases.
Related coverage
- SGLang and AntLing cut trillion-parameter restart time to 32 seconds
- Token fingerprints link anonymous Ox Alpha to Baseten’s GLM-5.2 Vision
- Ox Alpha reports an 80% pass rate on the full 113-task DeepSWE run