#distributedTraining

A deep dive into the hidden machinery behind distributed LLM training. #dataparallelism #distributedtraining...Show more

Join @clockworkio & @linuxfoundation for a free live webinar TOMORROW at 9:00 AM PT: "Handling Hardware Failures During Training: A Comparative Analysis of Fault Tolerant Training Frameworks". Learn more & register: #OpenSource #Linux #DistributedTraining #FaultTolerance #MLInfrastructure #MLOps

Thrilled to be speaking and teaching at @NVIDIAGTC 2026! When I was new to ML, organizing @WomenWhoCode workshops in Delhi and nerding out over NLP papers-I couldn't have imagined I'd one day be on stage at GTC talking about training models reliably at scale. But here we are🧵

Resilient training isn't anymore a nice-to-have — in my opinion it's table stakes and the next thing in industry (already a now thing for some). I am excited to democratize my 2 years of learnings with you all very soon! If you're scaling distributed training, come say hi.

Even if you can't make it to the talk or the course and are attending GTC, DM me! Hyped for everything that is to come!! 🧷 Register: See you in San Jose! March 16-19. #GTC26 #NVIDIA #DistributedTraining #NVRx #NeMo #AI #FaultTolerance #NVIDIAGTC

🧵 Exciting new features to scale ML/AI efficiently! We just released key updates focusing on compute infrastructure: [@]huggingface_hub for loading/caching large models [@]model for storing & registering models [@]checkpoint for robust training jobs

Train with confidence! With the new [@]checkpoint decorator, you can checkpoint training jobs, ensuring you don’t lose progress due to failures. Automatic resumption and checkpoint tracking save time and money! 💰 #PyTorch #MLTraining

Distributed training just got easier on Outerbounds! Now generally available, it supports multi-GPU setups with [@]torchrun and [@]metaflow_ray. Perfect for large data and models. Train efficiently, even at scale! 💡 #AI #DistributedTraining

Learn how to provision a multi-node GPU training cluster on @CrusoeAI with Terraform - specifically a 2-node A100 setup with InfiniBand: #DistributedTraining #Terraform #MLOps #Crusoe

Day 27 of #100DaysOfCode: 🌟 Today, I coded for Distributed Training! 🖥️ 🔄 Even without multiple CPUs, I learned how to scale deep learning models across devices. Excited to apply this knowledge in future projects! #DistributedTraining #DeepLearning #AI

Hiring now: Distributed Training Engineer, Sora Apply ➜ OpenAI 📍 San Francisco 💰 Salary negotiable #distributedtraining #openaijobs #hiring #jobsearch #techjobs

Communication is often the bottleneck in distributed AI. Gensyn’s CheckFree offers a fault-tolerant pipeline method that yields up to 1.6× speedups with minimal convergence loss. @gensynai #AI #DistributedTraining

Train large language models in a shared and sustainable way? Join #Bittensor subnet 38 #DistributedTraining and start mining #Tao.

RT Cost Efficient Distributed Training with Elastic Horovod and Amazon EC2 Spot Instances #elastic #amazonec2 #distributedtraining #horovod #deeplearning

✅ Achievement Unlocked! Efficient #DistributedTraining of DeepSeek-R1 671B is now a reality on #openEuler 24.03! 💙 Dive in and see what this #TechCP, openEuler + #DeepSeek, bring to the table 👉 #opensource #ValentinesDay2025 #ValentinesDay #openEulerBuzz

As in #DistributedTraining the models are trained over a distributed system, there are several benefits of such approach 👇 Full article:

Scaling up training with 50TB of data? Here's why our machine learning engineer Ahmad Sachal chose WebDataset for efficient processing and reduced costs! 🚀💻 Find out how we optimized our data loading in this article: #MLOps #DistributedTraining

What should you consider while choosing the right tools for #DistributedTraining? Article by @nielspace07 (best frameworks & tools that offer DT inside):

A use for forward-forward learning: distributed training? #distributedtraining

In #DistributedTraining, the training workload is divided across multiple processors, called workers. These workers are trained in parallel to speed up model training. There are 2 approaches to parallelism: - data parallelism - model parallelism More:

🔥 Scaling LLMs with Distributed Training: Learn how data and model parallelism techniques maximize the resource utilization and reduce training costs. #MachineLearning #DistributedTraining #DataParallelism #ModelParallelism 🔥

Yes, @Michael_D_Moor, @Pseudomanifold, and I have a WhatsApp group called “gym pull && push”. And if we commit, we push. I was sick, so my branch is behind several commits. But I’ll merge myself back into the gym. Thanks @github #VersionOfYourselfControl #DistributedTraining

This is worth a deeper look! #TensorFlow #distributedTraining #nn

Uber open-sourced Horovod, a framework for fast distributed TensorFlow. Seems Uber uses Keras & writes custom CUDA?

Distributed Training using Apache MXNet with Horovod #DistributedTraining #Apache #MXNet #Horovod #Tensorflow #Keras #PyTorch #deepLearning #framework #Uber #ApacheMXNet