The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs With Hybrid Parallelism

Yosuke Oyama, Naoya Maruyama, Nikoli Dryden, Erin McCarthy, Peter Harrington, Jan Balewski, Satoshi Matsuoka, Peter Nugent, Brian Van Essen. The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs With Hybrid Parallelism. IEEE Trans. Parallel Distrib. Syst., 32(7):1641-1652, 2021. [doi]

Abstract

Abstract is missing.