Route CloudWatch and Datadog alerts to on-call

Most teams already have alerts in three places: CloudWatch for AWS, Alertmanager for Kubernetes, and some SaaS monitor on top. Each one pages in its own way, and none of them knows who is on call this week.

ZipLogger accepts alerts from those tools over a webhook and turns them into incidents. They get the same deduplication, on-call routing and escalation as ZipLogger's own alert rules. The responses shown below come from running real Alertmanager and CloudWatch payloads through ZipLogger's ingest code.

The ten sources

Go to Alerts → Detection → Sources and click + New integration. Pick a Provider:

Under Routing you set a Default service, Default environment and Default severity for when the payload doesn't say, plus an Escalation policy or an On-call team (when no policy). Leave both empty and incidents are recorded but nobody is paged. The on-call schedules post covers building the policy.

Click Create and show credentials. You get the webhook URL, and a secret if the auth mode needs one. Both are shown once. ZipLogger keeps only a hash of the key and an encrypted secret. An Admin can use Rotate secret later, which stops the old URL immediately.

The URL always has this shape:

https://app.ziplogger.ai/api/v1/integrations/webhooks/YOUR_INTEGRATION_KEY

The key starts with zli_. The URL is the first secret, so treat it like a password even in "secret URL only" mode.

Prometheus Alertmanager

Add a receiver to alertmanager.yml and route to it. With the Bearer token header mode:

receivers:
  - name: ziplogger
    webhook_configs:
      - url: 'https://app.ziplogger.ai/api/v1/integrations/webhooks/YOUR_INTEGRATION_KEY'
        send_resolved: true
        max_alerts: 100
        http_config:
          authorization:
            credentials: 'YOUR_SECRET'

route:
  receiver: ziplogger

max_alerts: 100 is there on purpose. ZipLogger refuses a notification that carries more than 100 alerts, and a refusal drops the whole delivery. With the cap, Alertmanager trims the list instead.

Each alert in the notification becomes its own incident, keyed by Alertmanager's fingerprint. Kubernetes labels are read: namespace/pod becomes the resource, service, app or deployment the service, cluster or env the environment, and severity the severity.

We sent a KubePodCrashLooping alert for pod shop/checkout-api-7d9f8-x2k4q four ways. These are the responses:

firing       -> 202 {"accepted":1,"note":null,"results":[{"kind":"Trigger","title":"KubePodCrashLooping: Pod is crash looping","outcome":"Created","incidentId":"0310af92-..."}]}
firing again -> 202 {"accepted":1,"note":null,"results":[{"kind":"Trigger","title":"KubePodCrashLooping: Pod is crash looping","outcome":"Duplicate","incidentId":null}]}
resolved     -> 202 {"accepted":1,"note":null,"results":[{"kind":"Resolve","title":"KubePodCrashLooping: Pod is crash looping","outcome":"Resolved","incidentId":"0310af92-..."}]}
wrong token  -> 401 {"error":"Invalid bearer token.","code":"auth"}

The second delivery is Alertmanager re-sending the same alert at its repeat interval. It is recognised and changes nothing, so nobody is paged twice. The incident ended up with severity critical, service checkout-api (from the app label, not the kube-state-metrics job), environment prod-eu, and the Prometheus graph link as its source link.

AWS CloudWatch through SNS

This route needs no AWS credentials in ZipLogger. Create the integration with provider AWS CloudWatch and Secret URL only, then:

1. In SNS, create a topic (or reuse the one your alarms notify) and add an HTTPS subscription with the ZipLogger URL. Leave raw message delivery off.
2. ZipLogger confirms the subscription by itself. It only ever contacts sns.*.amazonaws.com for this.
3. Point each alarm's ALARM and OK actions at the topic. Without the OK action nothing resolves automatically.

Every SNS message's signature is checked against Amazon's certificate. Responses from the run, for an alarm on HTTPCode_Target_5XX_Count of an ALB:

SubscriptionConfirmation -> 200 {"confirmed":true,"subscription":"arn:aws:sns:us-east-1:123456789012:alarms:sub-1"}
ALARM                    -> 202 {"accepted":1,"note":null,"results":[{"kind":"Trigger","title":"CloudWatch alarm: checkout-alb-5xx","outcome":"Created","incidentId":"f4383103-..."}]}
same SNS message again   -> 202 {"accepted":1,"note":null,"results":[{"kind":"Trigger","title":"CloudWatch alarm: checkout-alb-5xx","outcome":"Duplicate","incidentId":null}]}
INSUFFICIENT_DATA        -> 202 {"accepted":0,"note":"Alarm "checkout-alb-5xx" is INSUFFICIENT_DATA; only ALARM and OK change an incident.","results":[]}
OK                       -> 202 {"accepted":1,"note":null,"results":[{"kind":"Resolve","title":"CloudWatch alarm: checkout-alb-5xx","outcome":"Resolved","incidentId":"f4383103-..."}]}
tampered message         -> 401 {"error":"SNS signature does not match the message.","code":"sns_signature"}

The incident's description is the alarm's NewStateReason plus its description. The resource is the first metric dimension, here app/checkout-alb/50dc6c495c0c9188, and the source link opens the alarm in the CloudWatch console. CloudWatch alarms carry no severity, so the integration's Default severity is used. If you want critical and warning alarms treated differently, use two topics and two integrations.

Two limits: the SNS message must be CloudWatch's default alarm JSON, not a custom text message. And EventBridge works too, through an API destination with the Bearer token header mode and a rule matching "detail-type": ["CloudWatch Alarm State Change"].

Datadog

In Datadog, open Integrations → Webhooks and create a webhook with the ZipLogger URL. With the bearer mode, add a custom header {"Authorization": "Bearer YOUR_SECRET"}. Use this payload, which Datadog fills in:

{
  "id": "$ID", "alertId": "$ALERT_ID", "transition": "$ALERT_TRANSITION",
  "title": "$ALERT_TITLE", "message": "$EVENT_MSG", "priority": "$ALERT_PRIORITY",
  "type": "$ALERT_TYPE", "metric": "$ALERT_METRIC", "query": "$ALERT_QUERY",
  "scope": "$ALERT_SCOPE", "tags": "$TAGS", "hostname": "$HOSTNAME",
  "org": "$ORG_NAME", "link": "$LINK", "date": "$DATE"
}

Then mention @webhook-<name> in each monitor that should page. The monitor id plus its group scope is the dedupe key. service: and env: tags become the incident's service and environment, and P1 to P5 map to critical through info.

Anything else: the generic webhook

For a tool that has no adapter, post JSON. The default mode signs each request with the secret:

BODY='{"action":"trigger","title":"Nightly export is late","severity":"high","service":"exports","dedupeKey":"export-late","eventId":"export-2026-09-29"}'
TS=$(date +%s)
SIG=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac 'YOUR_SECRET' | sed 's/^.* //')
curl -X POST 'https://app.ziplogger.ai/api/v1/integrations/webhooks/YOUR_INTEGRATION_KEY' \
  -H 'Content-Type: application/json' \
  -H "X-ZipLogger-Timestamp: $TS" -H "X-ZipLogger-Signature: v1=$SIG" \
  --data "$BODY"

Send "action":"resolve" with the same dedupeKey to close it. A request whose timestamp is more than five minutes off is refused: 401 {"error":"Timestamp outside the allowed window of 300 seconds.","code":"auth"}.

How dedup and auto-resolve work

• One open incident per alert. A trigger with the same dedupe key as an open incident is added to it instead of opening a new one. The key is the provider's own identity where there is one: the alarm ARN, the Alertmanager fingerprint, the Datadog monitor and scope, the dedupeKey you send.
• Keys are per integration. The same key arriving through two integrations makes two incidents.
• Retries are free. A delivery with an event id ZipLogger has already processed returns Duplicate and changes nothing.
• Recovery resolves. OK, resolved or Recovered closes the open incident and stops any escalation in flight. A recovery with nothing open is recorded against the last incident with that key, or ignored.

Each integration card has Send test alert and Send test recovery buttons, which push a real event through the same pipeline. The card also counts accepted and rejected deliveries and shows the Last problem, which is the first place to look when a tool says it sent something.

Pushing incidents out to PagerDuty or Opsgenie

If your rota lives in PagerDuty or Opsgenie, you can keep it. Go to Alerts → Delivery → Destinations and click + Connect a destination (Admin only).

• PagerDuty: paste an Integration key (Events API v2) from the service you want paged.
• Opsgenie: paste the API key of an API integration and pick the Region, US or EU.

The key is checked against the provider before it is stored. Then choose Minimum severity, and optionally a Service and Environment. With Send matching incidents automatically on, every matching incident is raised there too. Acknowledging here acknowledges there and Resolving here resolves there keep the two in step.

For the other direction, click Issue inbound key and paste that URL into PagerDuty's Generic Webhooks (v3) or Opsgenie's Webhook integration. Acknowledging or closing the alert over there then marks the ZipLogger incident the same way. PagerDuty deliveries are signature-checked. Opsgenie sends no signature, so its URL is the only credential.

The external alert's dedupe key is always ziplogger:<incident id>, so a retry can never open a second PagerDuty incident.

What it does not do

• Push only. ZipLogger never polls CloudWatch, Datadog or any other API. If the tool doesn't send, nothing arrives.
• Only the formats listed. Azure must use the common alert schema. CloudWatch must use the default alarm JSON.
• No triggers from PagerDuty. An incident opened in PagerDuty that ZipLogger didn't create stays in PagerDuty.
• Size and rate limits. 256 KB per delivery, 100 alerts per batch, and 600 deliveries a minute per integration key before 429.

Next step

Every provider's setup steps are shown in the app under Setup instructions on its integration card. The incident management page explains what happens to an incident after it opens, and creating a free workspace takes a minute. Add a source, press Send test alert, and check the incident opens where you expect.