An anomaly alert fired 40 times last month, 38 of them noise. Redesign the detection rule so the team trusts it again.
This rule fires whenever today's signups fall more than 20% below the single value from 7 days ago. It produced 40 alerts last month, 38 of them false. Constraints: signups have strong weekly seasonality, occasional holidays, and low-volume days are noisy.
# Fires if today's signups fall >20% below the value 7 days ago
def is_anomaly(today, week_ago):
return today < 0.8 * week_agoRewrite the rule so it stops firing on noise while still catching real drops.
Replace the single-point threshold with a seasonal, robust, persistent rule: baseline against prior same-weekdays, not one point from 7 days ago. Flag only when the deviation clears a robust spread of a few MAD, persists past one interval, and beats a minimum absolute effect.
- ✗Loosening the threshold instead of adding seasonality and persistence
- ✗Comparing to a single reference point rather than a distribution
- ✗Firing on one reading with no minimum absolute effect
- →How would you tune the persistence and minimum-effect thresholds?
- →How do you keep the rule from missing a slow real decline?
The naive rule compares today to a single point 7 days ago: it catches weekly seasonality (that point could itself be a holiday), explodes on noisy low-volume days, and fires off one reading. The robust replacement: baseline against a window of prior same-weekdays, measure deviation in MAD units (outlier-resistant), and add persistence plus a minimum absolute effect.
import numpy as np
def is_anomaly(today, same_weekday_history, min_abs_drop=50, k=4):
# same_weekday_history: signups on prior same weekdays (e.g. last 8 Saturdays)
med = np.median(same_weekday_history)
mad = np.median(np.abs(same_weekday_history - med)) or 1.0
robust_z = (today - med) / (1.4826 * mad)
return robust_z < -k and (med - today) >= min_abs_drop
def fire_alert(today_series):
# require the breach to persist over >1 consecutive interval
return all(is_anomaly(*args) for args in today_series[-2:])The seasonal baseline removes the weekly pattern and holidays; the MAD threshold lets normal wobble pass; min_abs_drop silences low-volume noise; the 2-interval check kills one-off blips. A real, sustained drop still clears every condition.
