Densifying Assumed-Sparse Tensors - Improving Memory Efficiency and MPI Collective Performance During Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

Derya Cavdar, Valeriu Codreanu, Can Karakus, John A. Lockman III, Damian Podareanu, Vikram A. Saletore, Alexander Sergeev, Don D. Smith II, Victor Suthichai, Quy Ta, Srinivas Varadharajan, Lucas A. Wilson, Rengan Xu, Pei Yang. Densifying Assumed-Sparse Tensors - Improving Memory Efficiency and MPI Collective Performance During Tensor Accumulation for Parallelized Training of Neural Machine Translation Models. In Michèle Weiland, Guido Juckeland, Carsten Trinitis, Ponnuswamy Sadayappan, editors, High Performance Computing - 34th International Conference, ISC High Performance 2019, Frankfurt/Main, Germany, June 16-20, 2019, Proceedings. Volume 11501 of Lecture Notes in Computer Science, pages 23-39, Springer, 2019. [doi]