Back to questions
JuniorTheoryCommon
What does this program print?
import numpy as np
from sklearn.tree import DecisionTreeRegressor
X = np.array([[1], [2], [3], [4]])
y = np.array([2.0, 4.0, 10.0, 12.0])
pred = np.full(4, y.mean())
tree = DecisionTreeRegressor(max_depth=1, random_state=0).fit(X, y - pred)
pred = pred + 0.5 * tree.predict(X)
print(pred[0], pred[3])5.0 9.0
- Expecting
3.0 11.0: the tree's leaf values-4.0and4.0are multiplied by the learning rate0.5before they are added, so the mean7.0moves by only2.0. - Expecting
4.5 9.5: a depth-1 tree has two leaves, each predicting the mean residual of its samples, so-5and-3both become-4, not each its own value.
- What would the second tree be trained on after this step?
- Why does the first prediction of the ensemble start from the mean of
y?
