Declare incidents automatically from other tools

Turn signals from your monitoring, ticketing, and internal tools into incidents, without anyone declaring them by hand.

There are two ways to declare incidents from another tool: send its events to an HTTP alert source, or call the create incident API. Which one you want comes down to whether anyone has decided yet that this is an incident.

• Use an alert source when the tool is reporting a signal that might be an incident: a monitor firing, an error rate climbing, a check failing. We’ll deduplicate repeated events, group related alerts into one incident, page the right people, and can decline the incident again if the signal clears.
• Use the API when someone or something has already made the call: a support agent hitting Escalate in your ticketing tool, a form in an internal portal, or a script running your own logic. You get an incident straight away, with whatever fields you set.

And if your tool is already one of our supported alert sources, start there. You get all the alert source behavior without building anything.

Send alerts to an alert source

Create an HTTP alert source. It gives you a URL and a secret token to send events to:

curl --request POST 'https://api.incident.io/v2/alert_events/http/<ALERT_SOURCE_ID>' \
  --header 'Authorization: Bearer <ALERT_SOURCE_TOKEN>' \
  --header 'Content-Type: application/json' \
  --data '{
    "title": "Checkout error rate above 5%",
    "status": "firing",
    "deduplication_key": "checkout-error-rate",
    "description": "5xx rate has been above 5% for 2 minutes.",
    "source_url": "https://monitoring.example.com/alerts/123",
    "metadata": { "service": "checkout", "team": "payments" }
  }'

If you can’t change the shape of what your tool sends, use a custom HTTP source with a transform expression instead. See the alert events API for every field.

How events become alerts

The deduplication_key identifies the thing you’re alerting on, and it drives everything else:

• Sending firing again with the same key updates the existing alert rather than creating a new one, so a monitor that re-sends every minute still only produces one alert.
• Send the same key with "status": "resolved" when the signal clears, and the alert resolves with it.
• If it fires again after that, you get a new alert, because the previous one is already resolved.

Use metadata to set alert attributes like the affected service or team, so you can route and filter on them. Each alert source has its own rate limit.

How alerts become incidents

An alert route connects the alert source to incidents and paging. For each route, you choose:

• Which alerts count, by filtering on alert attributes and priority.
• Whether related alerts share an incident, by grouping alerts that fire close together or share attributes like the service.
• Whether incidents start in triage, so a responder can accept or decline before the incident process kicks in. Tick Decline triage incidents if the linked alerts are resolved and we’ll decline them for you when the signal clears.
• Who gets paged, through escalation paths. Paging and incident creation are independent, so a route can page without creating an incident, or the other way round.
• How the incident looks: its name, summary, severity, and custom fields, all set from the alert. See Incident templates.

Declining or resolving the incident resolves its alerts. During a maintenance window, matching alerts skip their alert routes, and the window decides what happens to them instead.

Call the create incident API

Create an API key with the Create incidents permission, and call the create incident endpoint:

curl --request POST 'https://api.incident.io/v2/incidents' \
  --header 'Authorization: Bearer <YOUR_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "idempotency_key": "zendesk-ticket-48213",
    "visibility": "public",
    "name": "Customers can't complete checkout",
    "severity_id": "<SEVERITY_ID>"
  }'

• Use an ID from your own tool as the idempotency_key, like the ticket number. If you send the same key again you get back the incident you already created, so retries can never create duplicates.
• The incident starts in an active status and opens its own Slack or Microsoft Teams channel, unless its incident type is set up to start in triage or skip the channel. Active incidents need a severity_id, and you can list yours with the severities endpoint.
• You can set anything a responder would: incident type, custom fields, role assignments, timestamps. Creating your first incident using the API walks through it.
• To run workflows on incidents created this way, add a condition on the API Key creator. That lets you do things like invite the support agent who escalated the ticket.

Creating incidents through the API has its own rate limit, which is lower for incidents that open a new channel. If the tool might send you a burst, as a monitor would, that’s another reason to use an alert source.

Page someone without an incident

To page a team without declaring an incident, create an escalation through the API, or use an alert route that escalates without creating incidents.