Protocol Learning Workshop | Pluralis Research

Protocol Learning

Training frontier foundation models today demands massive, co-located clusters of high-end GPUs - accessible only to a handful of the most well-resourced organizations. Protocol Learning removes this co-location requirement, enabling multi-participant training of foundation models across open, permissionless networks of globally distributed compute, where no single participant has, or can ever obtain, a full copy of the model.

This requires solving hard open problems in low-bandwidth model parallelism, asynchronous distributed optimization, supporting heterogeneous hardware, fault-tolerant training systems, Byzantine robustness, and trustless verification. This workshop convenes the researchers advancing these building blocks to define the challenges ahead and chart a research roadmap for training the next generation of community-owned frontier models with self-sustaining economics.

Photos

Talks

Dr. Alexander Longualis ResearchUnextractable Protocol Models](https://www.youtube.com/watch?v=_2_S2qzsNXU)Prof. Nic LaneUniversity of Cambridge / Flower LabsWhy AI's Compute Scarcity Is Artificial](https://www.youtube.com/watch?v=KEkbzPrWXeQ)Dr. Max RyabininTogether AICommunication-Efficient Training in the Era of Reinforcement Learning](https://www.youtube.com/watch?v=JCbUOCSEjj0)Prof. Namhoon LeePOSTECHMitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation](https://www.youtube.com/watch?v=gR4McN9VnYo)Dr. Sameera RamasinghePluralis ResearchCommunication-Efficient Model-Parallel Training](https://www.youtube.com/watch?v=2L41M5ZGxiI)Dr. Oğuzhan ErsoyGensynAttacks and Defenses in Decentralised Collaborative Learning](https://www.youtube.com/watch?v=LgXRw9XcS3Q)Riccardo PatanaPluralis Research, ex-AnthropicBeyond Open Weights, Collectively Built, Trustlessly Owned, Sovereign by Design](https://www.youtube.com/watch?v=UUfR0d3Obhs)

Lightning Talks

Rishi SharmaEPFL (PhD)Shattered but Stronger: Model Fragmentation for Privacy and Efficiency in Decentralized Learning](https://www.youtube.com/watch?v=ylzTdHvjrwo)Benjamin ThérienUniversité de Montréal (PhD Student)Overcoming the Communication-Performance Tradeoff in LLM Pre-training](https://www.youtube.com/watch?v=1TQe2T6DSSc)Arto MaranjyanEPFL (Postdoctoral Researcher)Controlling Delay in Asynchronous SGD: Optimality for Any Data Regime](https://www.youtube.com/watch?v=gdR0agmkEtY)Andrej JovanovićUniversity of CambridgeDistributed Training via Local Updates and Infrequent Communication](https://www.youtube.com/watch?v=-oB9l_6MmeU)Alan AboudibMacrocosmosResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism](https://www.youtube.com/watch?v=lH4OmukWTS4)Kaja GruntkowskaKAUST Center of Excellence for Generative AI (PhD)Error Feedback for Muon and Friends](https://www.youtube.com/watch?v=ATuPgNExUs0)