Tag: training
All the articles with the tag "training".
Parallelism Strategies in Distributed Computing
One map of DP, DDP, FSDP, ZeRO, TP, PP, CP, SP and EP across training and inference, what each costs, and which ones vLLM supports.
All the articles with the tag "training".
One map of DP, DDP, FSDP, ZeRO, TP, PP, CP, SP and EP across training and inference, what each costs, and which ones vLLM supports.