Skip to main content

Anomaly Detection

Two methods, one pattern: rolling baseline + threshold. For each event, compute a rolling average and standard deviation; if the event's value is more than sigma * sd away from the mean, flag it.

  • baseline(col, opts) — appends avg / sd / upper / lower columns to the source schema. Drives band charts and outlier filters.
  • outliers(col, opts) — sugar over the same rolling pass, returns just the anomalous events.

Both are built on top of rolling(). Reach for them directly when the rolling-baseline pattern is what you want; reach for rolling() itself for arbitrary windowed reductions.

baseline(col, opts)

const bands = cpu.baseline('cpu', { window: '1m', sigma: 2 });
// bands.schema: [time, cpu, host, avg, sd, upper, lower]

Four optional number columns appended:

  • avg — rolling average of col
  • sd — rolling standard deviation of col
  • upperavg + sigma * sd
  • loweravg - sigma * sd

Then a band chart is two toPoints calls; an outlier filter is one:

const data = bands.toPoints();
// [{ ts, cpu, ..., avg, sd, upper, lower }, ...] — chart-ready

const anomalies = bands.filter((e) => {
const v = e.get('cpu');
const lo = e.get('lower');
const hi = e.get('upper');
return v != null && lo != null && hi != null && (v > hi || v < lo);
});

Warm-up and flat windows

  • Before the rolling window has enough samples for a meaningful baseline, avg and sd are undefined; upper and lower follow.
  • When the rolling window is flat (every sample identical), sd === 0. avg and sd still report their real values, but upper and lower collapse to undefined — a zero-width band would flag every non-equal point as anomalous, which is usually the opposite of what you want.

Filters that null-check upper / lower will do the right thing through warm-up and flat-window regions.

The default warm-up only suppresses output until there's at least one sample. On noisy sub-second telemetry that's usually too eager — a 2- or 3-sample window has a near-zero rolling stdev, so the band collapses tight around the first few values and false-flags the next normal-looking event. Pass minSamples to widen the warm-up:

cpu.baseline('cpu', { window: '1m', sigma: 2, minSamples: 20 });

While the window holds fewer than minSamples source events, avg, sd, upper, and lower are all undefined. The option threads through to the internal rolling pass and is also available on outliers().

Custom column names

If the defaults (avg / sd / upper / lower) collide with source columns:

cpu.baseline('cpu', {
window: '1m',
sigma: 2,
names: { avg: 'cpuAvg', sd: 'cpuSd', upper: 'cpuHi', lower: 'cpuLo' },
});

Collisions throw at construction, so the rename is mandatory, not optional.

outliers(col, opts)

Just the anomalies — same schema as the source, filtered to events outside the band:

const anomalies = cpu.outliers('cpu', { window: '1m', sigma: 2 });
// anomalies.length ≤ cpu.length, schema unchanged.

Composes with the rest of the API for dashboards:

// Bucketed anomaly count per 15s for a bar chart.
const perBucket = anomalies.aggregate(Sequence.every('15s'), {
cpu: 'count',
});

// Per-host breakdown.
const perHost = anomalies.groupBy('host');

Conceptually equivalent to baseline(col, opts).filter(...); implemented independently (one rolling pass either way). Reach for baseline() directly when you want the band columns for charting. The two share flat-window semantics — both skip events where sd === 0.

Picking one

  • Drawing a band chart: baseline — you'll use the upper / lower columns directly.
  • Just want the spike list (alerting, counting, per-host): outliers.
  • Both at once: baseline — one rolling pass for everything.

Beyond rolling-baseline

baseline and outliers are deliberately the only first-party anomaly-detection helpers. Other patterns — STL decomposition, seasonal-adjusted thresholds, classifier-based detection — fit map, rolling, and reducer combinations cleanly enough that folding them into the library would just add a vocabulary tax. If you need one and the composition feels heavy, file an issue with the shape and we'll see if it's worth lifting.