This blog dives into a theoretical machine learning concept called the bias-variance decomposition. This decomposition is a method which examines the expected generalization error for a given learning algorithm and a given data source. This helps us understand various questions about model. Written by RAPIDS.
Generalization concerns overfitting or the ability of a model learned on training data to provide effective predictions on new unseen examples. We perform experiments using two popular tree ensemble learning algorithms, Gradient Boosting and Random Forests.
The main points of the article:
- Bias Variance decomposition explained
- Experimental Methodology
- Gradient Boosting
- Random Forests
Model performance may deteriorate based on two distinct sources of error -– bias and variance. Gradient boosting models combat both bias and variance by boosting for many rounds at a low learning rate. Further resources and charts include. Nice one!
[Read More]