Which Neural Net Architectures Give Rise to Exploding and Vanishing Gradients?

Boris Hanin. Which Neural Net Architectures Give Rise to Exploding and Vanishing Gradients?. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada. pages 580-589, 2018. [doi]

@inproceedings{Hanin18,
  title = {Which Neural Net Architectures Give Rise to Exploding and Vanishing Gradients?},
  author = {Boris Hanin},
  year = {2018},
  url = {http://papers.nips.cc/paper/7339-which-neural-net-architectures-give-rise-to-exploding-and-vanishing-gradients},
  researchr = {https://researchr.org/publication/Hanin18},
  cites = {0},
  citedby = {0},
  pages = {580-589},
  booktitle = {Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada},
  editor = {Samy Bengio and Hanna M. Wallach and Hugo Larochelle and Kristen Grauman and Nicolò Cesa-Bianchi and Roman Garnett},
}