July 2026

## [RL Post-Training on Macs](/content/blog/rl-post-training-on-macs/index.html)

Erfan Miahi

Post-training an 8B MoE with GRPO on 14 consumer Macs and one B200, decoupled over R2; held-out pass@1 on agentic search more than doubled, from 29% to 63%

July 2026

## [Factored Gossip DiLoCo: Reducing Blocking Communication in DiLoCo](/content/blog/factored-gossip-diloco/index.html)

Chamin Hewa Koneputugodage

[ICML 2026](https://icml.cc/virtual/2026/poster/66683) · [Paper](https://arxiv.org/abs/2606.22768)

Reducing blocking communication in DiLoCo by factoring synchronization into a non-blocking parameter mixing and a minimal blocking gradient mixing

March 2026

## [From Base to Reasoning Model: A Full Post-Training Pipeline on a Single Node](/content/blog/from-base-to-reasoning-model/index.html)

Shamane Siri

We replicated Meta's full post-training pipeline on a single compute node with one researcher, matching Llama-3.2-1B-Instruct quality

January 2026

## [Pluralis' Multi-party Training Stack](/content/blog/pluralis-multi-party-training-stack/index.html)

Pluralis Team

[Code](https://github.com/PluralisResearch/node0)

A deep dive into our library built for fault-tolerant multi-party distributed training

November 2025

## [Asynchronous Pipeline Parallelism](/content/blog/asynchronous-pipeline-parallelism/index.html)

J. Snewin, T. Ajanthan

[ICML 2025](https://icml.cc/virtual/2025/poster/46245) · [Paper](https://arxiv.org/pdf/2505.01099) · [Code](https://github.com/PluralisResearch/AsyncPP)

Nesterov Method for Asynchronous Pipeline Parallel Optimization

May 2025

## [SWARM Parallel with Asynchronous Updates](/content/blog/swarm-parallel-async-updates/index.html)

Yan Zuo, Gil Avraham

We significantly improve training reliability, robustness and speed of asynchronous pipeline-parallel training

May 2025

## [Beyond Top-K: Pipeline Parallelism Over Slow Networks](/content/blog/beyond-top-k-pipeline-parallelism/index.html)

Sameera Ramasinghe

A novel method enabling efficient model-parallel training over low-bandwidth networks with 90% compression

April 2025

## [Efficient Asynchronous Low-Bandwidth Training on Heterogenous GPUs](/content/blog/efficient-asynchronous-low-bandwidth/index.html)

Thalaiyasingam Ajanthan

A new asynchronous method that surpasses synchronous methods in low-communication training while supporting heterogenous GPUs

March 2025

## [A Third Path: Protocol Learning](/content/blog/a-third-path-protocol-learning/index.html)

Alexander Long

Developing the true open-source AI

October 2024

## [Protocol Learning, Protocol Models and the Great Convergence](/content/blog/protocol-learning-protocol-models/index.html)

Alexander Long

Two enormous, previously disparate fields converge and a path towards the largest models to ever be trained is opened

July 2024

## [Decentralized Training Looms](/content/blog/decentralized-training-looms/index.html)

Alexander Long

Collaborative Training of foundation models is closer to actualization than broadly understood. The popular view that low bandwidth node-to-node connections render this infeasible is incorrect