Exploiting Input Tensor Dynamics in Activation Checkpointing for Efficient Training on GPU

Jianjin Liao, Mingzhen Li, Hailong Yang, Qingxiao Sun, Biao Sun, Jiwei Hao, Tianyu Feng, Fengwei Yu, Shengdong Chen, Ye Tao, Zicheng Zhang, Zhongzhi Luan, Depei Qian. Exploiting Input Tensor Dynamics in Activation Checkpointing for Efficient Training on GPU. In IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023, St. Petersburg, FL, USA, May 15-19, 2023. pages 156-166, IEEE, 2023. [doi]

Abstract

Abstract is missing.