XGBoost: Enhancement Over Gradient Boosting Machines
In the first part of this discussion on XGBoost, I set the foundation for understanding the basic components of boosting. In brief, boosting uses sequences of decision trees that seek to reduce the residuals of the prior tree. In other words, each new tree uses the... Read more
XGBoost is Machine Learning’s Captain America
Captain America is great in many ways. He is not necessarily the strongest, nor the fastest. He cannot fly or shoot anything our his hands. However, he is a consistent leader, that is well understood, admired and very effective. Also, when needed he can even wield... Read more
Regression Blog 2: We’re Practically Giving These Regressions Away
When I heard that they would be releasing Pumpkin Spice Spam, I thought of regression. This might seem like a leap, but bear with me. In the U.S. in the last few years, hearing news reports of unusual Pumpkin Spice flavored products means the unofficial end of... Read more
Some Details on Running xgboost
While reading Dr. Nina Zumel’s excellent note on bias in common ensemble methods, I ran the examples to see the effects she described (and I think it is very important that she is establishing the issue, prior to discussing mitigation). [Related Article: When Less is More: A Brief Story About Feature... Read more
Getting Up to Speed with XGBoost in R
In this article we’ll take a brief tour of the XGBoost package in R. Previously, I wrote an overview article “Gradient Boosting and XGBoost” which you might wish to read first in order to get some background before digesting the code in this article. As a... Read more