LibGuides: DDS-8536: Module 2 Resources

Module 2 Required Readings

The Elements of Statistical Learning: Data Mining, Inference, and Prediction
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer.
Read Chapters 10 and 15, which provide a detailed explanation of bagging, boosting, and Random Forests. These chapters are essential for understanding the theoretical foundation of ensemble methods and how they improve model performance by reducing bias and variance.

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow
Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow
Géron, A. (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems (2nd ed.). O'Reilly Media. https://learning.oreilly.com/library/view/hands-on-machine-learning/9781098125967/preface01.html#idm45720251756320
Focus on Chapter 7, which introduces Random Forests and Ensemble Learning techniques such as bagging and boosting with practical Python examples. Géron also covers XGBoost and includes useful code snippets to help implement these methods in practice.
Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow
Raschka, S., & Mirjalili, V. (2019). Python machine learning: Machine learning and deep learning with Python, scikit-learn, and TensorFlow (3rd ed.). Packt Publishing.
https://learning.oreilly.com/library/view/python-machine-learning/9781789955750/?sso_link=yes&sso_link_from=nationaluniversity
Read Chapter 7, which is dedicated to Combining Different Models for Ensemble Learning. This chapter provides practical implementations of bagging and boosting techniques in Python, including AdaBoost, XGBoost, and Random Forests. This resource will be useful when applying ensemble methods during the assignments.

XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794. https://doi.org/10.48550/arXiv.1603.02754
This paper presents the design and implementation of XGBoost, a highly efficient boosting algorithm that students will be applying in this module. Reviewing this paper will provide insight into how XGBoost works and how to optimize its performance for large datasets.

Bagging Predictors
Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123-140.
https://link.springer.com/content/pdf/10.1023/A:1018054314350.pdf
This foundational paper explains bagging (Bootstrap Aggregating) and its effectiveness in reducing variance and improving the robustness of models. Students will use this paper to understand the theoretical foundations behind Random Forests and other bagging techniques applied in the module.
Pattern Recognition and Machine Learning
Bishop, C. M. (2006). Pattern recognition and machine learning. Springer.
https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf
Focus on Chapter 14, which covers Ensemble Learning Methods. This chapter provides a theoretical understanding of how multiple models can be combined to create stronger predictors, supporting the practical techniques introduced in Module 2.

Deep Learning
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
https://www.deeplearningbook.org/
Read Chapter 7, which provides insights into regularization techniques. While this chapter focuses on deep learning, the regularization techniques discussed are also relevant to boosting and bagging methods, particularly when tuning models to avoid overfitting.