JuniorTheoryCommon

What does this program print?

Python

import numpy as np

x = np.array([1.0, 2.0, 3.0])
y = np.array([2.0, 4.0, 6.0])
w, b, lr = 0.0, 0.0, 0.1
dw = 2 * ((w * x + b - y) * x).mean()
db = 2 * (w * x + b - y).mean()
w = w - lr * dw
b = b - lr * db
print(round(w, 2), round(b, 2))

1.87 0.8

Expecting -1.87 -0.8: w - lr * dw steps against the gradient; at w = b = 0 every prediction is below its target, so dw and db are negative and both parameters grow.
Expecting 18.67 8.0: the step is the gradient times lr, so the gradients -18.67 and -8.0 shrink tenfold before they are subtracted.

What happens to w over many steps if the learning rate lr is set to 1.0 here?
Why do both gradients come out negative at the starting point w = 0, b = 0?

Back to questions Open topic in roadmap →Answer it yourself — Quiz

Back to questions

JuniorTheoryCommon

What does this program print?

Python

import numpy as np

x = np.array([1.0, 2.0, 3.0])
y = np.array([2.0, 4.0, 6.0])
w, b, lr = 0.0, 0.0, 0.1
dw = 2 * ((w * x + b - y) * x).mean()
db = 2 * (w * x + b - y).mean()
w = w - lr * dw
b = b - lr * db
print(round(w, 2), round(b, 2))

1.87 0.8

Expecting -1.87 -0.8: w - lr * dw steps against the gradient; at w = b = 0 every prediction is below its target, so dw and db are negative and both parameters grow.
Expecting 18.67 8.0: the step is the gradient times lr, so the gradients -18.67 and -8.0 shrink tenfold before they are subtracted.

What happens to w over many steps if the learning rate lr is set to 1.0 here?
Why do both gradients come out negative at the starting point w = 0, b = 0?

Back to questions Open topic in roadmap →Answer it yourself — Quiz