Implement OnlineClassifier, a binary classifier trained in batch and then updated in real time, with SGD and without sklearn. There is no randomness at all: the weights always start at 0, making this fully deterministic (the same discipline as Mini Recommender L1's bias-only baseline and Mini Search Ranker's linear model). This problem builds one full lifecycle of a real online classification system: batch training → streaming fine-tuning → concept-drift detection → probability calibration.

Shared spec

model = OnlineClassifier(n_features=3, epochs=20, lr=0.1, reg=0.01)
model.fit(X)                    # X: list[(features: list[float], label: 0|1)]
model.predict_proba(features)   # float in [0, 1]
model.predict(features)         # 0 or 1

Linear-sigmoid scoring formula

$z = \sum_{i=0}^{n-1} w_i x_i + b, \qquad p = \frac{1}{1 + e^{-z}}$

The weight vector w (length n_features, all starting at 0.0) and the bias b (starting at 0.0) stay untouched until fit is called. With w and b all 0, z=0, so predict_proba is always 0.5 — including before fit.

predict(features) is 1 when predict_proba(features) >= 0.5 and 0 otherwise — an exact tie at 0.5 resolves to 1.

Training — stochastic gradient descent on log loss

Repeat epochs times. Each epoch walks X once in its original order (no shuffling). For each (features, label):

p    = predict_proba(features)      # computed with the pre-update w, b
err  = label - p                    # the log-loss gradient falls out in exactly this
                                     # form — not a lucky coincidence that it matches
                                     # squared-error SGD's (actual - predicted), but the
                                     # standard result of differentiating the sigmoid
                                     # and log loss together.

for i in range(n_features):
    w[i] <- w[i] + lr * (err * features[i] - reg * w[i])
b      <- b + lr * err

⚠️ There is no regularization term on b. Each w[i] decays by - reg * w[i], but b updates from err alone — the same deliberate asymmetry as Mini Search Ranker's b rule. Regularizing b like w produces different results.

Compute err once from the w and b at the start of this step, and use that one value to update all of w and b together (do not update w[i] first and then use the new value when computing w[i+1] or b).

An empty X finishes fit without raising, leaving w and b at 0. Re-fitting the same model discards the previous w and b entirely and starts again from 0 (it does not accumulate). fit returns self so it can be chained.

Streaming fine-tuning — partial_fit

partial_fit(features, label) -> self

Applies exactly one step of fit's SGD inner loop (with the same self.lr and self.reg) directly to w and b. There is no notion of epochs — one example, one step. It shares the very same w and b as fit, so calling partial_fit(...) after fit(...) shows up immediately in the next predict/predict_proba. Unlike fit, partial_fit resets nothing — it is purely cumulative and incremental.

Concept-drift detection — a sliding accuracy window

model = OnlineClassifier(n_features, epochs, lr, reg,
                          drift_window=0, drift_threshold=None)
model.observe_and_partial_fit(features, label)   # bool — did this call just detect drift?
model.drift_detected     # sticky — once True, stays True until reset_drift()
model.reset_drift()

• With drift_window=0 (the default), drift detection is entirely off — observe_and_partial_fit still performs the partial_fit update, but it always returns False and self.drift_detected is False forever (the same "optional feature, off by default" convention as every other problem here, a regression safeguard).
• With drift_window > 0, maintain a sliding window (size drift_window, evicting the oldest once full) of the prediction accuracy of the last drift_window observe_and_partial_fit calls (predict(features) == label, scored before this example's partial_fit update — score first, then update, the same ordering discipline as Mini Anomaly Detector's observe).
• Compute the window's accuracy (correct / drift_window) only when the window is full. If that accuracy is below drift_threshold (<) and self.drift_detected is not already True → set self.drift_detected = True and return True from this call. When the window is not full, drift was already detected, or the accuracy is at or above drift_threshold, this call returns False.
• self.drift_detected is sticky — once True it stays True until you call reset_drift(), and in the meantime observe_and_partial_fit keeps returning False (it has already reported).
• reset_drift() sets drift_detected back to False and also clears the accuracy window — that is what makes drift detection genuinely start over, so re-detection requires a fresh window to fill from scratch. Resetting only the flag while leaving the stale window behind is the common wrong answer: drift fires again on the very first call after the reset.

Platt calibration — calibrate / predict_calibrated

model.calibrate(X_cal)              # X_cal: the same shape as fit's X
model.predict_calibrated(features)  # float in [0, 1]

• Platt scaling trains a one-dimensional logistic regression mapping the raw model's $z = \sum w_i x_i + b$ (frozen at the w and b current when calibrate is called — the main w and b are never updated again during calibration) onto the true labels. Its parameters are a scalar weight A and bias B (both starting at 0.0, entirely separate from the main w/b and never touching them), trained by SGD.

• Walk each (features, label) of X_cal in the given order, repeating for a fixed calibration epoch count of 50 (a constant, not a parameter) and updating with a fixed calibration learning rate of 0.1 (also a constant):

  textCopy

  z    = the main model's raw z (w, b frozen as of the start of calibrate)
  p    = 1 / (1 + exp(-(A*z + B)))
  err  = label - p
  A    <- A + 0.1 * err * z
  B    <- B + 0.1 * err
  

  There is no regularization — calibration training is deliberately simple.

• predict_calibrated(features) computes the main model's raw z and returns 1 / (1 + exp(-(A*z + B))). If calibrate was never called, A=0.0 and B=0.0, so predict_calibrated is always 0.5 regardless of the main model's state (that falls out of the formula, it is not a special branch).

• An empty X_cal finishes calibrate without raising and leaves A and B exactly as they were (they are not reset to 0) — a genuine no-op. This is a deliberate contrast with fit, which resets w and b every time.

Level structure

Each level must preserve the behavior of the previous ones. With drift_window=0, observe_and_partial_fit behaves exactly like a plain wrapper around partial_fit even after L3 (a regression safeguard).

Start with the batch training that forms the online classifier's skeleton. w and b start at 0, so everything is deterministic — no randomness required.

OnlineClassifier(n_features: int, epochs: int, lr: float, reg: float)
fit(X: list[tuple]) -> self       # X: list[(features: list[float], label: 0|1)]
predict_proba(features: list[float]) -> float
predict(features: list[float]) -> int      # 0 or 1

• predict_proba is z = sum(w[i]*features[i] for i in range(n_features)) + b followed by p = 1 / (1 + exp(-z)).
• predict is 1 when predict_proba(features) >= 0.5 and 0 otherwise — an exact tie at 0.5 resolves to 1.
• fit follows the shared spec's SGD rules exactly — walk X in its original order epochs times, and at each step compute err = label - p from the pre-update w and b, then update all of w and b together. There is no regularization term on b — only each w[i] decays by - reg * w[i].
• Empty X finishes without raising, leaving w and b at 0.
• Re-fitting the same model discards the previous w and b entirely and starts again from 0 (nothing accumulates).
• Before fit is called, predict_proba is always 0.5 (w and b are 0, so z=0).
• fit returns self so it can be chained.