How catastrophic can catastrophic forgetting be in linear regression?

Itay Evron, Edward Moroshko, Rachel Ward, Nathan Srebro, Daniel Soudry. How catastrophic can catastrophic forgetting be in linear regression?. In Po-Ling Loh, Maxim Raginsky, editors, Conference on Learning Theory, 2-5 July 2022, London, UK. Volume 178 of Proceedings of Machine Learning Research, pages 4028-4079, PMLR, 2022. [doi]

@inproceedings{EvronMWSS22,
  title = {How catastrophic can catastrophic forgetting be in linear regression?},
  author = {Itay Evron and Edward Moroshko and Rachel Ward and Nathan Srebro and Daniel Soudry},
  year = {2022},
  url = {https://proceedings.mlr.press/v178/evron22a.html},
  researchr = {https://researchr.org/publication/EvronMWSS22},
  cites = {0},
  citedby = {0},
  pages = {4028-4079},
  booktitle = {Conference on Learning Theory, 2-5 July 2022, London, UK},
  editor = {Po-Ling Loh and Maxim Raginsky},
  volume = {178},
  series = {Proceedings of Machine Learning Research},
  publisher = {PMLR},
}