[DOC]: why all examples are running at one node? #3634

Activity

1. LarryZhangy

   added

   documentationImprovements or additions to documentationImprovements or additions to documentation

   on Apr 24, 2023
2. LarryZhangy

   changed the title [-][DOC]: why all examples is running at one node?[/-] [+][DOC]: why all examples are running at one node?[/+] on Apr 24, 2023
3. JThh commented on Apr 24, 2023

   

   JThh

   on Apr 24, 2023

   Last edited by JThh

   Contributor

   More actions

   This is somehow limited to our dev environment. But we are definitely supportive of multi-node training. You need to set -nnodes=x for running, and make sure your nodes are managed by the nccl backend.

   👍React with 👍3Reacted by LarryZhangy, suc16 and 梁爽

   🚀React with 🚀1Reacted by LarryZhangy
4. JThh commented on Apr 24, 2023

   

   JThh

   on Apr 24, 2023

   Contributor

   More actions

   Actually, there was an issue regarding this: #2921.

   👍React with 👍2Reacted by LarryZhangy and 梁爽
5. JazyJiang commented on Oct 19, 2026

   

   JazyJiang

   on Oct 19, 2026

   Collaborator

   More actions

   Closing together with #2921: multi-node launching is documented in https://colossalai.org/docs/basics/launch_colossalai (colossalai run --hostfile ... or SLURM), and the examples can be run across nodes that way.
6. JazyJiang

   closed this as completedon Oct 19, 2026