#NCCL

1 week ago

#NCCL watchdog timeouts are often misunderstood. Meta’s analysis shows >60% are caused by CPU-side stuckness or divergence, not the network. This guide explains using #FlightRecorder to trace collective states and fix hangs

Read: https://bit.ly/4bCqItC #OpenSourceAI #PyTorch

0 0 0 0

HGPU group

@hgpu.bsky.social

4 months ago

GPU-Initiated Networking for NCCL Modern AI workloads, especially Mixture-of-Experts (MoE) architectures, increasingly demand low-latency, fine-grained GPU-to-GPU communication with device-side control. Traditional GPU communicatio…

GPU-Initiated Networking for NCCL

#CUDA #NCCL

hgpu.org?p=30411

0 0 0 0

Ahmet BÜTÜN

@ahmetbutun.bsky.social

8 months ago

Enhancing GPU Communication: Key Insights into NCCL Tuning Explore the significance of NCCL tuning for optimizing GPU-to-GPU communication in AI workloads. Learn how custom tuner plugins and strategic adjustments can enhance performance.

Enhancing GPU Communication: Key Insights into NCCL Tuning Explore the significance of NCCL tuning for optimizing GPU-to-GPU communication in AI workloads. Learn how custom tuner plugins and strategic adjustments can enhance... @cosmicmeta.ai #NCCL

https://u2m.io/7MzQyINO

0 0 0 0

Ahmet BÜTÜN

@ahmetbutun.bsky.social

9 months ago

NVIDIA Enhances Multi-GPU Communication with NCCL 2.26 Release NVIDIA's NCCL 2.26 introduces performance enhancements, improved monitoring, and quality of service features, optimizing multi-GPU and multinode communications for AI and HPC applications.

NVIDIA Enhances Multi-GPU Communication with NCCL 2.26 Release NVIDIA's NCCL 2.26 introduces performance enhancements, improved monitoring, and quality of service features, optimizing multi-GPU and multinode communications for AI... @cosmicmeta.io #NCCL

https://u2m.io/vPWS3wuE

0 0 0 0

@becs-j.bsky.social

11 months ago

@aafp.org I just heard Shawn Martin said social media is important at #NCCL but I need more activity on Blue Sky please!

1 1 0 0

Justin Huyck

@justinhuyck.bsky.social

7 years ago

ICYMI Early-bird Reg by Jan 15 for May @NCCLonline #NCCL #NCCL2019: "Nurturing the Hungry Heart." 1st day is "Longing": may we encounter God longing w/us through ecclesial pain & joy, ministerial exhaustion & energy, family turmoil & domestic discipleship

0 0 0 0

Justin Huyck

@justinhuyck.bsky.social

7 years ago

Looking forward to May #NCCL #NCCL2019 convocation: "Nurturing the Hungry Heart." Register by Jan 15 for discount. Daily Themes: Longing, Nurturing, Preparing, Commissioned.

0 0 0 0