Blog | Pluralis Research
July 2026
RL Post-Training on Macs
Erfan Miahi
Post-training an 8B MoE with GRPO on 14 consumer Macs and one B200, decoupled over R2; held-out pass@1 on agentic search more than doubled, from 29% to 63%
July 2026
Factored Gossip DiLoCo: Reducing Blocking Communication in DiLoCo
Chamin Hewa Koneputugodage
Reducing blocking communication in DiLoCo by factoring synchronization into a non-blocking parameter mixing and a minimal blocking gradient mixing
March 2026
From Base to Reasoning Model: A Full Post-Training Pipeline on a Single Node
Shamane Siri
We replicated Meta's full post-training pipeline on a single compute node with one researcher, matching Llama-3.2-1B-Instruct quality
January 2026
Pluralis' Multi-party Training Stack
Pluralis Team
A deep dive into our library built for fault-tolerant multi-party distributed training
November 2025
Asynchronous Pipeline Parallelism
J. Snewin, T. Ajanthan
Nesterov Method for Asynchronous Pipeline Parallel Optimization
May 2025
SWARM Parallel with Asynchronous Updates
Yan Zuo, Gil Avraham
We significantly improve training reliability, robustness and speed of asynchronous pipeline-parallel training
May 2025
Beyond Top-K: Pipeline Parallelism Over Slow Networks
Sameera Ramasinghe
A novel method enabling efficient model-parallel training over low-bandwidth networks with 90% compression
April 2025
Efficient Asynchronous Low-Bandwidth Training on Heterogenous GPUs
Thalaiyasingam Ajanthan
A new asynchronous method that surpasses synchronous methods in low-communication training while supporting heterogenous GPUs
March 2025
A Third Path: Protocol Learning
Alexander Long
Developing the true open-source AI
October 2024
Protocol Learning, Protocol Models and the Great Convergence
Alexander Long
Two enormous, previously disparate fields converge and a path towards the largest models to ever be trained is opened
July 2024
Decentralized Training Looms
Alexander Long
Collaborative Training of foundation models is closer to actualization than broadly understood. The popular view that low bandwidth node-to-node connections render this infeasible is incorrect