Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, Shen Li. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. PVLDB, 16(12):3848-3860, 2023. [doi]
Abstract is missing.