Pick a common SRE alert template, tune the PromQL and thresholds, add routing labels and Alertmanager annotations, and copy a ready-to-paste groups: YAML block for prometheus.yml or a Kubernetes PrometheusRule.
Start from a template, then tweak.
Templates can use {{ $labels.LABEL }}, {{ $value }}, and the humanize family (humanize, humanizePercentage, humanizeDuration, humanize1024).
–
–
Prometheus Operator manifest. The rule contents match the block above, plus a PrometheusRule wrapper with role: alert-rules, so the operator picks it up automatically.
expr & for. Prometheus evaluates expr at each interval. If it returns a non-empty vector, the alert enters pending, and after for keeps returning results it flips to firing. A short for catches transient spikes; too short means noisy pages.
keep_firing_for. Since Prometheus 2.42, this delays resolution so a flapping series doesn't oscillate between firing and resolved. Reasonable defaults: 0m (default), or 10m for on-call pages you want stable.
Labels vs. annotations. Labels form the alert identity and route the alert through Alertmanager (severity: critical, team: platform, …). Annotations are human context (summary, description, runbook_url, dashboard_url).