← Gym/Mini Experimentation Framework
00:00/ 55 min

🧭 Do not search for the first 15 minutes. When stuck: re-read the requirements → define I/O → choose the data structure → trace a small example by hand → write code.

Build a single Bandit class that both routes traffic across alternatives (arms) while learning — a multi-armed bandit — and decides whether the conversion-rate gap between two arms is statistically meaningful, an A/B significance test.

Shared spec

python
bandit = Bandit(n_arms=3, epsilon=0.1, seed=0, strategy="epsilon_greedy", c=2.0)

The constructor takes all five arguments from the start. strategy and c gain meaning at L2, record and is_significant at L3, traffic_split and winner at L4 — but the argument order never changes across levels.

Reward estimation (all levels)

Each arm holds a count (times selected) and a value (estimated mean reward). Both start at 0.

text
update(arm, reward):
    count[arm] += 1
    value[arm] += (reward - value[arm]) / count[arm]

This is an incremental mean — do not recompute sum/count. When several arms tie for the maximum, pick the lowest index (a rule that recurs throughout this problem).

Random-number protocol — used only by the epsilon_greedy strategy

python
self.rng = np.random.default_rng(seed)   # exactly once, in the constructor

Every call to select_arm() calls self.rng.random() exactly once first and compares it against epsilon.

  • < epsilon → explore: call self.rng.integers(0, n_arms) exactly once more and return that value as the arm.
  • otherwise → exploit: return the arm with the largest value (ties go to the lowest index). Do not make the second call in this case. Calling twice every time regardless of the branch shifts every subsequent draw and produces a completely different sequence.

Level structure

LevelAdded requirement
1select_arm and update — ε-greedy only
2strategy="ucb1" — a second selection strategy
3record and is_significant — conversion-rate A/B significance
4traffic_split and winner — declaring a winner and routing traffic

Each level must preserve the behavior of the previous ones.

Level 1 · ε-greedy selection

At this level treat strategy as always "epsilon_greedy" (ucb1 arrives at L2).

python
Bandit(n_arms: int, epsilon: float, seed: int, strategy="epsilon_greedy", c=2.0)
select_arm() -> int
update(arm: int, reward: float) -> None
  • Follow the shared spec's random-number protocol (one call decides explore vs. exploit; the second call happens only when exploring) and the incremental-mean update.
  • count and value all start at 0. When exploiting, ties for the maximum go to the lowest index.