TL;DR

This post serves as a study note on the topic of "overfitting" in machine learning. It would also help to clarify some common misunderstandings on this topic. Although "overfitting to noise" is common, it's not the root cause of the lack of generalization in overly complex models.