Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices

Max Ryabinin, Eduard Gorbunov, Vsevolod Plokhotnyuk, Gennady Pekhimenko. Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual. pages 18195-18211, 2021. [doi]

Authors

Max Ryabinin

This author has not been identified. Look up 'Max Ryabinin' in Google

Eduard Gorbunov

This author has not been identified. Look up 'Eduard Gorbunov' in Google

Vsevolod Plokhotnyuk

This author has not been identified. Look up 'Vsevolod Plokhotnyuk' in Google

Gennady Pekhimenko

This author has not been identified. Look up 'Gennady Pekhimenko' in Google