#DistributedTraining

Shoutout to @johannes_hage, co-founder and CTO of Prime Intellect! 🌍 He co-authored OpenDiLoCo and the INTELLECT-1, 2 and 3 reports, training open models across globally distributed compute. #AI #OpenSource #DistributedTraining

Hiring now: Distributed Training Engineer, Sora Apply ➜ OpenAI 📍 San Francisco 💰 Salary negotiable #distributedtraining #openaijobs #hiring #jobsearch #techjobs

A deep dive into the hidden machinery behind distributed LLM training. #dataparallelism #distributedtraining...Show more

Join @clockworkio & @linuxfoundation for a free live webinar TOMORROW at 9:00 AM PT: "Handling Hardware Failures During Training: A Comparative Analysis of Fault Tolerant Training Frameworks". Learn more & register: #OpenSource #Linux #DistributedTraining #FaultTolerance #MLInfrastructure #MLOps

Learn how to provision a multi-node GPU training cluster on @CrusoeAI with Terraform - specifically a 2-node A100 setup with InfiniBand: #DistributedTraining #Terraform #MLOps #Crusoe

✅ Achievement Unlocked! Efficient #distributedtraining of #DeepSeek-R1:671B is realized on #openEuler 24.03! Built for the future of #AI, openEuler empowers #developers to push the boundaries of innovation. 🐋Full technical deep dive coming soon! @deepseek_ai #opensource #LLM

🧵 Exciting new features to scale ML/AI efficiently! We just released key updates focusing on compute infrastructure: [@]huggingface_hub for loading/caching large models [@]model for storing & registering models [@]checkpoint for robust training jobs

Train with confidence! With the new [@]checkpoint decorator, you can checkpoint training jobs, ensuring you don’t lose progress due to failures. Automatic resumption and checkpoint tracking save time and money! 💰 #PyTorch #MLTraining

Distributed training just got easier on Outerbounds! Now generally available, it supports multi-GPU setups with [@]torchrun and [@]metaflow_ray. Perfect for large data and models. Train efficiently, even at scale! 💡 #AI #DistributedTraining

Communication is often the bottleneck in distributed AI. Gensyn’s CheckFree offers a fault-tolerant pipeline method that yields up to 1.6× speedups with minimal convergence loss. @gensynai #AI #DistributedTraining

Thrilled to be speaking and teaching at @NVIDIAGTC 2026! When I was new to ML, organizing @WomenWhoCode workshops in Delhi and nerding out over NLP papers-I couldn't have imagined I'd one day be on stage at GTC talking about training models reliably at scale. But here we are🧵

Resilient training isn't anymore a nice-to-have — in my opinion it's table stakes and the next thing in industry (already a now thing for some). I am excited to democratize my 2 years of learnings with you all very soon! If you're scaling distributed training, come say hi.

Even if you can't make it to the talk or the course and are attending GTC, DM me! Hyped for everything that is to come!! 🧷 Register: See you in San Jose! March 16-19. #GTC26 #NVIDIA #DistributedTraining #NVRx #NeMo #AI #FaultTolerance #NVIDIAGTC

Data from block 3849132 (a second ago) --------------- 💎🚀 Subnet32 emission has changed from 2.587461% to 2.5954647% #bittensor #decentralizedAI #distributedtraining $tao #subnet @ai_detection

Is your training dataset eating up the computing resources❓ Try #MindSpore's #DistributedTraining parameter server, to get a leg up on the competition!😎 Read today's blog to learn more, and stay tuned for more groundbreaking solutions.🙌 #MeetMindSpore #PoweredByMindSpore

Train large language models in a shared and sustainable way? Join #Bittensor subnet 38 #DistributedTraining and start mining #Tao.

Day 27 of #100DaysOfCode: 🌟 Today, I coded for Distributed Training! 🖥️ 🔄 Even without multiple CPUs, I learned how to scale deep learning models across devices. Excited to apply this knowledge in future projects! #DistributedTraining #DeepLearning #AI

✅ Achievement Unlocked! Efficient #DistributedTraining of DeepSeek-R1 671B is now a reality on #openEuler 24.03! 💙 Dive in and see what this #TechCP, openEuler + #DeepSeek, bring to the table 👉 #opensource #ValentinesDay2025 #ValentinesDay #openEulerBuzz

RT Cost Efficient Distributed Training with Elastic Horovod and Amazon EC2 Spot Instances #elastic #amazonec2 #distributedtraining #horovod #deeplearning

RT Training BERT at a University #distributedsystems #distributedtraining #opensource #deeplearning

A use for forward-forward learning: distributed training? #distributedtraining

Current hardware performance can barely keep up with the growing demands on computing power.🤔🤔 But with #DistributedTraining, #MindSpore has found a way forward.🤗🤗 👉Read how: #PoweredByMindSpore

Yes, @Michael_D_Moor, @Pseudomanifold, and I have a WhatsApp group called “gym pull && push”. And if we commit, we push. I was sick, so my branch is behind several commits. But I’ll merge myself back into the gym. Thanks @github #VersionOfYourselfControl #DistributedTraining

In #DistributedTraining, the training workload is divided across multiple processors, called workers. These workers are trained in parallel to speed up model training. There are 2 approaches to parallelism: - data parallelism - model parallelism More: