← Back to documentation

Failure-Rate Alerts

Set a per-endpoint failure-rate alert and optional automatic pause.

5 min read

A failure-rate alert counts the recent endpoint outcomes. It notifies the selected relay targets when the failure rate is equal to a limit or higher.

Concept#

For each endpoint with the alerts enabled, PayloadRelay counts the recent request outcomes in the configured time window, and it calculates this value:

Code Example
failureRatePct = (failed_requests / total_requests) * 100

The calculation uses these rules:

  • PayloadRelay counts a request in the window when its outcome becomes final, and not when the request arrives. A delivery that fails after its retries counts in the window in which it fails.
  • total_requests does not contain these outcomes: the synthetic test outcomes (TEST_ACCEPTED, TEST_REJECTED, TEST_RATE_LIMITED), PENDING, FAILOVER_TRIGGERED, PAUSED, WEBHOOK_INGESTION_DISABLED, and EMAIL_INGESTION_DISABLED. A PENDING request counts when its outcome becomes final.
  • failed_requests contains every failure outcome in total_requests. This includes the rejections at ingress, for example AUTH_FAILED, and the final failed deliveries. A completed outcome is not a failure.
  • After an alert fires, the next examination counts only the outcomes that became final after that alert.

PayloadRelay rounds the failure rate that it shows to two decimal places. The alert fires when the traffic reaches the minimum request count and the rounded rate reaches the threshold. The window must also have one failure as a minimum. So a threshold of 0 fires on any failure.

PayloadRelay does not examine a paused endpoint or a disabled endpoint.

Configuration#

FieldRangeDescription
EnabledOn / OffEnables or disables the failure-rate alert for this endpoint.
Failure-rate threshold0 to 100 %The rounded rolling failure rate that fires the alert. With 0, any failure fires the alert.
Window1 to 1440 minThe rolling time window for the outcome counts.
Minimum requests1 to 10,000The number of non-test requests that PayloadRelay needs before it examines the threshold. This prevents an alert for a low-volume endpoint.
Cooldown0 to 1440 minThe time in which PayloadRelay stops the duplicate alerts after an alert fires. With 0, the next examination can fire again while the condition continues.
Auto-pauseOn / OffPauses the endpoint for 30 minutes after an alert fires, if the delivery-failure rate also reaches the threshold. During the pause, the inbound traffic receives 503 and creates no Activity row.
Alert targetsA maximum of 25 relay targets in the same organizationThe relay targets that receive the alert notification. PayloadRelay does not support a Google Sheets target for an alert.

Configure the alerts in the endpoint editor.

Rules for the alert targets:

  • If the alert is enabled, select one alert target as a minimum, or enable auto-pause. Otherwise the save returns 400 with Select at least one alert target, or enable auto-pause.
  • An endpoint can already have an enabled alert with no target and no auto-pause. A save that keeps this configuration succeeds. That alert notifies nobody, and only the alert history records it.
  • An email relay target must be confirmed before you add it. An SMTP relay target must also be verified.
  • If an attached email relay target stops being ready, it stays attached when you save. PayloadRelay skips that target when it sends an alert.
  • If PayloadRelay skips every target, the alert still starts the cooldown. The alert history records 0 notified targets.

PayloadRelay keeps the alert history for 30 days.

Auto-pause behavior#

Auto-pause uses a second rate that counts only the deliveries:

Code Example
deliveryFailureRatePct = (DELIVERY_FAILED / (ACCEPTED + EMAIL_INGESTION_ACCEPTED + DELIVERY_FAILED)) * 100

When an alert fires, PayloadRelay calculates this rate for the same window. The rate uses the same threshold and the same minimum requests. If the rate reaches the threshold, PayloadRelay rejects the inbound relay traffic with 503 Service Unavailable for 30 minutes. A rejection at ingress, for example AUTH_FAILED, can fire the alert, but it cannot pause the endpoint.

The traffic during the pause creates no Activity row. Auto-pause does not make a longer manual pause shorter. You can resume the endpoint at any time.

Cooldown semantics#

After an alert fires, the configured cooldown stops the duplicate alerts. An alert can fire again after the cooldown while the rate stays at the threshold or higher.

Alert payload#

A webhook target receives this JSON payload directly:

Code Example
{
  "event": "payloadrelay.failure_alert",
  "endpointId": "uuid",
  "endpointName": "...",
  "failureRatePct": 87.50,
  "totalRequests": 16,
  "failedRequests": 14,
  "windowMinutes": 15,
  "autoPaused": true,
  "synthetic": false,
  "firedAt": "2026-04-25T13:00:00Z",
  "dashboardUrl": "https://app.payloadrelay.com/endpoints/details?endpointId=<id>"
}

An email target receives an HTML notification with the same fields. An email relay target in SMTP relay mode sends the alert through the SMTP server of that relay target.

A Slack, Discord, Microsoft Teams, PagerDuty, and Telegram target receives the same alert details in the delivery format of that destination. For Microsoft Teams, PayloadRelay puts the alert details in a MessageCard, and it puts the JSON details in the card text. For PagerDuty, see PagerDuty.

If the delivery of a live alert fails, PayloadRelay can send it again a maximum of 3 times. This applies to a webhook, Slack, Discord, Microsoft Teams, PagerDuty, or Telegram target. PayloadRelay does not retry a test alert.

Test alerts#

To send a synthetic alert to all the configured targets, use the Send test alert button in the editor. PayloadRelay marks a test alert as synthetic.

The test alert uses the saved alert configuration. If the alert configuration has unsaved changes, the button is not available. Save the endpoint first.

The test alert contains sample numbers that fire the saved configuration:

  • totalRequests is the Minimum requests value.
  • failedRequests is the smallest number of failures that reaches the threshold, with a minimum of 1.
  • failureRatePct is the rate of these two numbers. If no configured target can receive the alert, PayloadRelay rejects the test. If a temporary service problem stops the handoff, the request returns an error that you can retry. A successful response means that PayloadRelay gave the alert to one target as a minimum. It does not mean that a destination received the alert.
← Back to documentation