JuniorTheoryCommon

What does this program print?

Python

import numpy as np
from sklearn.linear_model import Ridge

X = np.array([[1.0], [2.0], [3.0], [4.0]])
y = np.array([2.0, 4.0, 6.0, 8.0])
weak = Ridge(alpha=0.01).fit(X, y)
strong = Ridge(alpha=100.0).fit(X, y)
print(round(weak.coef_[0], 2), round(strong.coef_[0], 2))

2.0 0.1

Expecting 2.0 2.0 because the points lie exactly on a line: the penalty grows with alpha and pulls the slope toward zero even when the fit is perfect.
Expecting 0.0 for the strong model: the L2 penalty of Ridge shrinks a coefficient but never makes it exactly zero, so it stays at 0.1.

How does the intercept of strong change, given that Ridge does not penalize it?
Which alpha would you expect to work best on new data, and how would you choose it?

Back to questions Open topic in roadmap →Answer it yourself — Quiz

Back to questions

JuniorTheoryCommon

What does this program print?

Python

import numpy as np
from sklearn.linear_model import Ridge

X = np.array([[1.0], [2.0], [3.0], [4.0]])
y = np.array([2.0, 4.0, 6.0, 8.0])
weak = Ridge(alpha=0.01).fit(X, y)
strong = Ridge(alpha=100.0).fit(X, y)
print(round(weak.coef_[0], 2), round(strong.coef_[0], 2))

2.0 0.1

Expecting 2.0 2.0 because the points lie exactly on a line: the penalty grows with alpha and pulls the slope toward zero even when the fit is perfect.
Expecting 0.0 for the strong model: the L2 penalty of Ridge shrinks a coefficient but never makes it exactly zero, so it stays at 0.1.

How does the intercept of strong change, given that Ridge does not penalize it?
Which alpha would you expect to work best on new data, and how would you choose it?

Back to questions Open topic in roadmap →Answer it yourself — Quiz