Efficient Large Models Fine-tuning on Commodity Servers via Memory-balanced Pipeline Parallelism

Yujie Liu, Zhiquan Lai, Weijie Liu, Wei Wang, Dongsheng Li 0001. Efficient Large Models Fine-tuning on Commodity Servers via Memory-balanced Pipeline Parallelism. In IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application, HPCC/DSS/SmartCity/DependSys 2023, Melbourne, Australia, December 17-21, 2023. pages 726-727, IEEE, 2023. [doi]

Abstract

Abstract is missing.