JuniorTheoryCommon

What does this program print?

Python

import numpy as np

counts = np.array([20, 2])
means = np.array([0.6, 0.4])
t = counts.sum()
scores = means + np.sqrt(2 * np.log(t) / counts)
print(int(np.argmax(scores)), int(np.argmax(means)))

1 0

Expecting 0 0, as if the bonus could not beat a higher mean: arm 1 has only 2 pulls, so its bonus is about 1.76 against 0.56 for arm 0, and its score wins.
Expecting the bonus to penalize the rarely pulled arm: it is added to the mean and is largest where the count is smallest, which is what sends UCB to explore arm 1.

After many more pulls of both arms, which arm will UCB prefer, and why?
How does UCB decide when an arm has never been pulled?

Back to questions Open topic in roadmap →Answer it yourself — Quiz

Back to questions

JuniorTheoryCommon

What does this program print?

Python

import numpy as np

counts = np.array([20, 2])
means = np.array([0.6, 0.4])
t = counts.sum()
scores = means + np.sqrt(2 * np.log(t) / counts)
print(int(np.argmax(scores)), int(np.argmax(means)))

1 0

Expecting 0 0, as if the bonus could not beat a higher mean: arm 1 has only 2 pulls, so its bonus is about 1.76 against 0.56 for arm 0, and its score wins.
Expecting the bonus to penalize the rarely pulled arm: it is added to the mean and is largest where the count is smallest, which is what sends UCB to explore arm 1.

After many more pulls of both arms, which arm will UCB prefer, and why?
How does UCB decide when an arm has never been pulled?

Back to questions Open topic in roadmap →Answer it yourself — Quiz