Alerting: monitors & notifiers

An Axiom Monitor is a saved APL query that runs on a fixed cadence and fires when its condition is met. A Notifier is the destination it fires to — Slack, email, PagerDuty, a webhook. In alchemy both are resources in your Stack, so a threshold change or a new on-call channel is a reviewable diff, not a console edit.

Everything below runs inside your Stack’s Effect.gen — see Setup for credentials and the Axiom overview for datasets like app-traces and app-logs used in the queries.

Create a notifier

import * as Axiom from "alchemy/Axiom";

const slack = yield* Axiom.Notifier("ops-slack", {
  name: "ops-channel",
  properties: {
    slack: { slackUrl: process.env.SLACK_WEBHOOK_URL! },
  },
});

Exactly one channel goes under properties — here slack with an incoming webhook URL. The resource’s id output is what monitors target.

Threshold: alert when a value crosses a line

yield* Axiom.Monitor("error-rate", {
  name: "High error rate",
  description: "Fires when error count exceeds 100/5m",
  type: "Threshold",
  aplQuery: `
    ['app-traces']
    | where status >= 500
    | summarize count() by bin_auto(_time)
  `,
  operator: "Above",
  threshold: 100,
  intervalMinutes: 5,
  rangeMinutes: 5,
  alertOnNoData: false,
  resolvable: true,
  notifierIds: [slack.id],
});

Every intervalMinutes, the monitor runs aplQuery over the last rangeMinutes and compares the aggregate against threshold using operator (Above, AboveOrEqual, Below, BelowOrEqual, or AboveOrBelow). resolvable: true sends a resolution notice when the value drops back under the line; alertOnNoData controls whether an empty result counts as a problem.

notifierIds: [slack.id] is the wiring: slack.id is an output attribute of the notifier flowing into the monitor’s inputs, so alchemy deploys the notifier first and fills in the real id.

MatchEvent: alert on every matching event

yield* Axiom.Monitor("panics", {
  name: "Service panic",
  type: "MatchEvent",
  aplQuery: `['app-logs'] | where message contains "panic:"`,
  intervalMinutes: 1,
  rangeMinutes: 1,
  notifierIds: [slack.id],
});

A MatchEvent monitor has no threshold — it fires for every event the query matches. Use it for rare, individually-actionable events like panics or failed deploys, where one occurrence is already worth a page.

AnomalyDetection: alert on deviation from baseline

yield* Axiom.Monitor("traffic-anomaly", {
  name: "Traffic anomaly",
  type: "AnomalyDetection",
  aplQuery: `['app-traces'] | summarize count() by bin_auto(_time)`,
  compareDays: 7,
  tolerance: 25, // %
  intervalMinutes: 15,
  rangeMinutes: 15,
  notifierIds: [slack.id],
});

Instead of a static threshold, the monitor learns a baseline from the same window over the previous compareDays days and fires when the current result deviates by more than tolerance percent. Good for “traffic looks wrong” signals where the right absolute number changes by hour and weekday.

Changing a monitor’s type triggers a replacement; every other prop updates in place.

Fan out to multiple notifiers

const pagerduty = yield* Axiom.Notifier("pagerduty", {
  name: "primary-oncall",
  properties: {
    pagerduty: { routingKey: process.env.PAGERDUTY_ROUTING_KEY!, token: "" },
  },
});

yield* Axiom.Monitor("error-rate", {
  // ...same props as above
  notifierIds: [slack.id, pagerduty.id],
});

notifierIds takes any number of ids, so one monitor can post to the team channel and page on-call at the same time. Notifier channels cover Slack, email (email: { emails: [...] }), PagerDuty, Opsgenie, Discord, Microsoft Teams, and generic webhooks.

Custom webhook with a templated body

yield* Axiom.Notifier("incident-webhook", {
  name: "incident.io",
  properties: {
    customWebhook: {
      url: "https://api.incident.io/v2/alert_events",
      headers: { "Content-Type": "application/json" },
      secretHeaders: { Authorization: `Bearer ${process.env.INCIDENT_TOKEN}` },
      body: '{"title": "{{.Monitor.Name}}", "status": "firing"}',
    },
  },
});

For anything without a built-in channel, customWebhook gives you full control over the request: static headers, secretHeaders for credentials, and a body template with access to monitor fields like {{.Monitor.Name}}.

Derived alert fields with VirtualField

yield* Axiom.VirtualField("status-class", {
  dataset: "app-traces",
  name: "status_class",
  description: "HTTP response class bucket",
  expression: 'strcat(tostring(toint(status / 100)), "xx")',
  type: "string",
});

A VirtualField is a saved APL expression that appears as a derived column on its dataset at query time. Define common computations — status classes, latency buckets, parsed JSON paths — once, instead of repeating the expression in every monitor and dashboard. A virtual field is bound to one dataset; changing the dataset triggers a replacement.

Monitors on that dataset can then query the derived column directly:

yield* Axiom.Monitor("5xx-rate", {
  name: "5xx rate",
  type: "Threshold",
  aplQuery: `
    ['app-traces']
    | where status_class == "5xx"
    | summarize count() by bin_auto(_time)
  `,
  operator: "Above",
  threshold: 50,
  intervalMinutes: 5,
  rangeMinutes: 5,
  notifierIds: [slack.id],
});

If the definition of “5xx” ever changes, you update the virtual field’s expression — every monitor and dashboard built on status_class follows.

Where next

Ingest — datasets, ingest tokens, and wiring OTLP endpoints into your Workers and Functions.
Dashboards — chart the same queries your monitors run.
Axiom overview — all Axiom resources at a glance.

Reference: