Email Ingestion

Convert an inbound email to JSON and send it to the endpoint destinations.

18 min read

Email ingestion converts an inbound email into a JSON relay payload. Send an email to the endpoint email address. PayloadRelay sends the payload to the configured destinations, with webhook, email, Slack, Discord, Microsoft Teams, PagerDuty, Telegram, and Google Sheets.

How email ingestion operates#

Enable email ingestion on an endpoint.
Save the endpoint. Copy the emailIngestionAddress that the API gives, above the configuration tabs.
When a sender sends an email to that address, PayloadRelay parses the email into a JSON payload.
PayloadRelay examines the email, records the quota and the Activity metadata, and starts the delivery to the configured destinations.
SMTP returns success after PayloadRelay accepts the email. A temporary service problem returns SMTP 451, and the mail server of the sender can send the email again.

SMTP success means that PayloadRelay accepted the message. It does not mean that every downstream target delivered it. Activity can stay PENDING until all the delivery attempts end.

Email payload format#

PayloadRelay converts the inbound email into this JSON structure. The to list comes from the To: header of the message. It does not contain the ingestion address when the message arrives by BCC or through a forwarding alias.

{
  "from": "[email protected]",
  "fromName": "Example Sender",
  "envelopeFrom": "[email protected]",
  "to": ["[email protected]"],
  "cc": ["[email protected]"],
  "bcc": [],
  "replyTo": "[email protected]",
  "subject": "Order confirmation #1234",
  "textBody": "Your order has been confirmed...",
  "htmlBody": "<html>...</html>",
  "date": "2024-01-15T10:30:00Z",
  "messageId": "<[email protected]>",
  "headers": {
    "X-Mailer": "Outlook 16.0",
    "X-Priority": "1"
  },
  "authResults": {
    "spf": "pass",
    "dkim": "pass",
    "dmarc": "pass"
  },
  "authResultDetails": [
    {
      "authservId": "mx.example.net",
      "method": "spf",
      "result": "pass",
      "properties": { "smtp.mailfrom": "[email protected]" }
    }
  ]
}

Included fields#

The sender, the recipients (to, cc, bcc), and the reply-to address
The subject line
The text body content and the HTML body content
The date and the message ID
The selected email headers

Excluded content#

PayloadRelay removes the attachments. It relays the text content only.
The payload does not contain binary content, an inline image, or an embedded file.

Body-as-payload mode#

By default, the envelope JSON above is the relay payload. To use the plain-text body as the payload, enable Use the plain-text email body as the relay payload on the endpoint Details tab. For example, use this mode when an automated system sends a JSON document by email.

When this mode is enabled, PayloadRelay parses the body with the payload format of the endpoint:

Payload format	Parsing
`JSON`	PayloadRelay parses `textBody` as JSON. It rejects an HTML-only email. It does not use `htmlBody` as a fallback.
`XML`	PayloadRelay forwards `textBody` as a raw XML text node.
`text/plain`	PayloadRelay sends `textBody` as a string with no change.
`FORM`, `QUERY_PARAMS`, `QUERY_PARAMS_BASE64_JSON`	PayloadRelay does not support these formats. It rejects the email as `EMAIL_INGESTION_REJECTED`.

If the parsing fails, for example, when the body is not valid JSON for a JSON endpoint, PayloadRelay rejects the email with MALFORMED_JSON_PAYLOAD. The mail server of the sender receives an SMTP 550 permanent failure.

Filters and field validations#

When body-as-payload mode is enabled, the request body filters and the field validations in the Filter tab also apply to the parsed email body. You can remove a field, make a field necessary, or examine a field, as you do for an HTTP request.

When body-as-payload mode is disabled, the filters and the field validations have no effect, because the metadata envelope has a fixed structure.

Use envelope fields as template variables#

When body-as-payload is enabled, the {{email.*}} namespace gives the envelope fields. These fields contain the sender, the subject, the recipients, and the headers. You can use them in a webhook custom body, a custom header, and an email subject or body. A template variable is a name that PayloadRelay replaces with a value. A dynamic Reply-To field for a primary email target can use a path such as email.replyTo or email.from.

{{email.from}}
{{email.subject}}
{{email.textBody}}, {{email.htmlBody}}
{{email.date}}, {{email.messageId}}, {{email.replyTo}}
{{email.to}}, {{email.cc}}, {{email.bcc}}: PayloadRelay writes an array as a string with commas between the values.
{{email.headers.X-Mailer}}, or a different header name

These variables are also available in envelope mode. A regular variable such as {{from}} and {{subject}} resolves against the payload.

A failover email dynamic Reply-To field uses the failover payload. In envelope mode, use replyTo. In body-as-payload mode, use a manual Reply-To or a fallback Reply-To when the replies must go to the original sender.

Enabling email ingestion#

Open the endpoint editor.
In the header of the endpoint editor, select Email ingestion endpoint in the Endpoint type list.
Save the changes.
Open the saved endpoint and copy the email ingestion address that the API gives, above the configuration tabs.

The endpoint must be enabled and have the type Email ingestion endpoint to accept an inbound email. An endpoint has one type. A change from webhook to email clears the webhook configuration, with the inbound authentication, the captcha, the IP limits, the CORS, and the required headers.

Quota and billing#

Each new accepted inbound email consumes one accepted event from the monthly quota, in the same way as an accepted HTTP relay. A completed second delivery that PayloadRelay identifies as a duplicate consumes no quota, and it starts no new set of deliveries. An email relay payload also uses the plan size limit. The limit is 256 KB on Sandbox and Solo, 512 KB on the Team trial and Team, and 1 MB on Scale. For a custom plan, the agreement of the organization sets the limit.

Activity log#

An email-sourced relay appears in the Activity tab with these outcome types:

Outcome	Meaning
`DELIVERY_QUEUE_UNAVAILABLE`	A temporary service problem stopped the acceptance. PayloadRelay asks the mail server of the sender to send the email again. The row contains metadata only.
`EMAIL_INGESTION_ACCEPTED`	All the tracked deliveries ended with no error. The dashboard puts this outcome in `Completed`. It can appear as `Pending` while the deliveries continue.
`EMAIL_INGESTION_REJECTED`	PayloadRelay rejected the email, because the ingestion is off, because PayloadRelay does not support the body-as-payload format, or because the body is empty or HTML-only where `text/plain` is necessary.
`AUTH_FAILED`	The SPF, DKIM, or DMARC enforcement failed, or an SMTP envelope sender that is not empty is not on the allowed-senders list.
`REQUEST_BODY_FILTER_FAILED`	The body-as-payload content did not agree with a request body filter.
`FIELD_VALIDATION_FAILED`	The body-as-payload content did not agree with a field validation rule.
`MALFORMED_JSON_PAYLOAD`	Body-as-payload mode needed JSON, but the email text body was not valid JSON.
`PAYLOAD_TOO_LARGE`	The relay payload from the email is larger than the payload size limit of the plan.

SPF / DKIM / DMARC enforcement#

PayloadRelay can make a trusted SPF, DKIM, and DMARC result necessary before it accepts an inbound email.

Each endpoint has three independent controls in the Security tab: Require SPF pass, Require DKIM pass, and Require DMARC pass. They are off by default.

The enforcement fails closed. A rule passes only with a trusted pass. PayloadRelay rejects mail with no result for the rule in the same way as mail with a failed result. Enable a control only when the senders use that authentication method.

The three rules do not bind the identity in the same way:

Toggle	Identity binding
`Require SPF pass`	Bound. The `smtp.mailfrom` identity must match the domain of the SMTP envelope sender. For a null sender, PayloadRelay accepts a pass that is tied to the reported `smtp.helo` identity.
`Require DKIM pass`	Not bound. A trusted `dkim=pass` is sufficient with any signing domain (`header.d`). A forwarder, a mailing list, or an ESP can sign with its own domain. This proves that a system signed the message. It does not prove that the message came from the From domain.
`Require DMARC pass`	Bound. When the message gives `header.from`, it must match the header From domain.

Use Require DMARC pass when the authentication must bind to the visible From address. Require DKIM pass alone does not prove the sender identity.

When the enforcement is enabled, PayloadRelay accepts a qualifying pass. A missing result, none, neutral, softfail, fail, and permerror make an SMTP 550 permanent failure. If there is no qualifying pass and a trusted result is temperror, PayloadRelay returns SMTP 451, and the mail server of the sender can send the email again.

Header result	Enforcement outcome
qualifying `pass`	PayloadRelay accepts the email.
`fail`, `softfail`, `permerror`	PayloadRelay rejects the email.
`none`, `neutral`	PayloadRelay rejects the email permanently.
`temperror` (with no qualifying pass)	PayloadRelay rejects the email temporarily. The sender receives SMTP `451`.
one rule in more than one trusted header	PayloadRelay discards the rule and uses it as a missing rule. PayloadRelay rejects the email.
missing header	PayloadRelay rejects the email.

PayloadRelay also sends the compatibility authResults object in the email envelope payload, for example, {{email.authResults.spf}}. authResultDetails contains each trusted result and the identity properties for the enforcement. The SPF smtp.mailfrom value must match the SMTP envelope domain. The DMARC header.from value, when the message gives it, must match the header From domain.

In one Authentication-Results header, a qualifying pass wins. More than one DKIM signature is alternative evidence, and one valid signature is sufficient. In that header, temperror wins against the permanent non-pass results in the summary.

The same rule in two separate trusted headers has a different result. A boundary mail server reports each rule one time. A second occurrence means that one header or more did not come from that server. PayloadRelay cannot identify the genuine header. It discards the rule from authResults and authResultDetails. The rule then operates as a missing rule, and an endpoint that needs it rejects the message.

The duplicate rule causes this result. The result values can be the same. A forged header that copies the genuine verdict and changes the identity can satisfy the identity binding, because SPF and DMARC accept a qualifying pass.

Trusted mail authentication results#

Use only trusted authentication results for the SPF, DKIM, and DMARC enforcement. If the organization uses custom mail routing, contact Support before you enable the enforcement. Support can make sure that the trusted results will be available.

SMTP envelope sender filter#

You can filter the SMTP envelope address with SMTP envelope sender filter in the Security tab.

Add a specific address: [email protected]
Use a wildcard for a pattern: *@company.com matches a sender from that domain.
To permit every sender, leave the list empty. This is the default.

Each entry must be a valid email address, or a wildcard pattern of the same shape, such as *@company.com, billing@*, or *@*.example.com. You can add a maximum of 100 unique entries. Each entry can have a maximum of 320 characters. PayloadRelay changes the entries to lowercase when you save them. PayloadRelay rejects an invalid entry and a duplicate entry with 400 Bad Request, before it saves the endpoint. PayloadRelay does not accept a universal *@* pattern. Leave the list empty instead.

If a sender is not empty and is not on the permitted list, PayloadRelay rejects the email with AUTH_FAILED in Activity. The mail server of the sender receives an SMTP 550 permanent failure. Activity records a null-sender DSN or bounce that an allowlist rejects as EMAIL_INGESTION_REJECTED.

PayloadRelay applies this filter to the SMTP envelope sender (MAIL FROM), and not to the From: header of the message. This filter alone does not authenticate the sender, because a sender can state the local part of the SMTP mailbox. Enable the SPF enforcement when the envelope domain must bind to a trusted SPF pass. SPF authenticates the domain, and not the local part of the mailbox.

PayloadRelay rejects a message with MAIL FROM:<[email protected]> and a forged From: [email protected] header when [email protected] is on the allowlist. PayloadRelay examines [email protected] only.
A DSN bounce message uses the SMTP null sender (MAIL FROM:<>). PayloadRelay rejects it when an allowlist is configured. To receive the bounces, leave the allowlist empty and use the SPF, DKIM, and DMARC enforcement.

DSN and bounce messages#

A Delivery Status Notification (DSN) and an automatic bounce message use the SMTP null sender MAIL FROM:<>. The examples are an out-of-office reply and a non-delivery report. RFC 5321 gives this sender. The SMTP envelope sender is an empty string.

With an allowlist: PayloadRelay rejects a null-sender message, because the empty sender matches no pattern. To receive a bounce notification or a DSN notification, leave the allowlist empty.

With no allowlist: a null-sender message passes the sender rule. The envelopeFrom field is an empty string. The from field contains the value of the From: header, which can be the address of the bounce originator.

With the SPF enforcement: for a null sender, PayloadRelay accepts a trusted SPF pass that is tied to the reported smtp.helo identity. Some DSN messages contain no trusted result. Use a separate endpoint with no SPF enforcement when the DSN delivery must not depend on that result.

Email metadata headers injected into the payload#

PayloadRelay converts each relayed email into a structured JSON payload. The top-level fields are always present. Some values can be null or empty.

Field	Source	Notes
`from`	The bare mailbox in `From:`	A sender can forge this value. For a trust-sensitive test, use the structured authentication results.
`fromName`	The name in `From:`	`null` when the header has no name.
`envelopeFrom`	SMTP `MAIL FROM`	An empty string for a DSN or bounce null sender. Use this field for the allowed-sender enforcement.
`to`	`To:` header	An array of addresses.
`cc`	`Cc:` header	An array of addresses. An empty array when the header is absent.
`bcc`	`Bcc:` header	Usually absent, because a delivered email does not normally contain this header.
`replyTo`	The effective reply address	Uses the first `Reply-To:` mailbox when it is present. If it is not present, uses the first `From:` mailbox. It is `null` only when the message has neither.
`subject`	`Subject:` header	An empty string when the header is absent.
`textBody`	First `text/plain` MIME part	`null` when the message has no plain-text part.
`htmlBody`	First `text/html` MIME part	`null` when the message has no HTML part.
`date`	`Date:` header	An ISO 8601 UTC string, for example, `2024-01-15T10:30:00Z`, or `null`.
`messageId`	`Message-ID:` header	Contains the angle brackets, such as `<[email protected]>`. If the header is absent, PayloadRelay creates a value, and the field is not null. The value applies to one delivery, and it is not stable. A retry of the same message makes a different value. Do not use it to deduplicate.
`headers`	All the other headers	A map from a header name to its first value. PayloadRelay removes the standard headers. See the list that follows.
`authResults`	`Authentication-Results:` header	A map with the keys `spf`, `dkim`, and `dmarc`, and values such as `pass` and `fail`.
`authResultDetails`	Trusted `Authentication-Results:` headers	An array of `{authservId, method, result, properties}`. `null` means that the detailed results are not available. An empty array means that the message reported no trusted result.

The `headers` map#

PayloadRelay sends all the non-standard headers in the headers map. It removes these standard headers, because they are top-level fields:

From, To, Cc, Bcc, Reply-To, Subject, Date, Message-ID, Content-Type, Content-Transfer-Encoding, MIME-Version, and Authentication-Results

PayloadRelay also removes the reserved X-PayloadRelay-Loop loop-protection header.

The headers map can contain these headers, and more:

Received: the SMTP relay trace headers that each hop adds. When the message has more than one, the map keeps the first upstream occurrence only. PayloadRelay removes its own receiving hop. A message that comes directly to PayloadRelay with no earlier hop has no Received entry.
DKIM-Signature: the raw DKIM signature from the sending domain.
Return-Path: the bounce address when the sending infrastructure added one. This header is usually absent, because a final delivery agent normally adds it.
ARC-*: the Authenticated Received Chain headers (RFC 8617).
X-Mailer, X-Priority, X-Spam-Status, and the other X- headers from the sender.
List-Unsubscribe, List-Id, and the other RFC 2369 list headers.

The Received, DKIM-Signature, and ARC-* headers can contain routing details and infrastructure details. PayloadRelay can send them to the downstream targets. If this is not permitted, use a transform rule to remove them before the delivery.

The `authResults` map#

The authResults map contains the trusted results that came with the message. The keys are spf, dkim, and dmarc. The values are lowercase result strings: pass, fail, softfail, neutral, none, temperror, and permerror. A key is absent when the message reported no trusted result for that rule.

You can use these values in a template as {{email.authResults.spf}}, {{email.authResults.dkim}}, and {{email.authResults.dmarc}}.

authResultDetails keeps more than one SPF result or DKIM result. If a rule is in more than one trusted header, PayloadRelay removes every entry for that rule, and no entry can satisfy the enforcement. The related properties are smtp.mailfrom, smtp.helo, header.d, and header.from. PayloadRelay uses the authentication results for the enforcement only after it makes sure that they are trustworthy.

Limits and security boundaries#

The inbound message size: the limit is approximately 2 MB. The front-end mail server accepts a maximum of 2,000,000 bytes. The receiver accepts a maximum of 2 MiB. PayloadRelay refuses a payload above the lower limit during the SMTP transaction. The attachments count against the limit, although PayloadRelay removes them.
The plan payload limits also apply. The email envelope JSON must stay in the plan limit. In body-as-payload mode, the text body must stay in the same limit. The limit is 256 KB on Sandbox and Solo, 512 KB on the Team trial and Team, and 1 MB on Scale. For a custom plan, the agreement of the organization sets the limit.
The attachments: the payload contains the text body and the HTML body only. See Multipart parsing.
The SPF, DKIM, and DMARC enforcement uses the trusted authentication results in the email. When a necessary result is not available, PayloadRelay rejects the message with the policy above.
The webhook security features: the inbound authentication, the captcha, the required headers, the IP allowlists, and the CORS apply to an HTTP request. They are not available for an email endpoint. Use the allowed senders and the sender authentication enforcement.
The transforms and the append-field rules apply to the JSON envelope or to the parsed body. They apply before the delivery, and you can copy values from textBody or headers into the outgoing payload.

Temporary payload handling#

PayloadRelay does not store a customer relay request body or a customer message body in its databases, activity logs, object storage, or backups. A body stays only in memory and in the delivery, retry, and dead-letter queues. It stays there only while PayloadRelay delivers the event. If the delivery fails permanently, the message stays in a dead-letter queue for a maximum of 7 days for diagnosis or a new delivery. PayloadRelay can keep operational metadata, such as an SMTP envelope sender address, in the 30-day Activity trail.

Delivery, retries, and deduplication#

Email ingestion and downstream delivery are at-least-once. PayloadRelay can identify a completed second delivery of an email, and prevent a quota charge and a new set of deliveries. A sender and a receiver must accept duplicates. A temporary SMTP 451 response asks the mail server of the sender to send the email again. That server controls the retry time and the retry duration.

The SMTP response tells the mail server of the sender if a new attempt is useful:

SMTP result	Typical conditions
`250`	New accepted mail, or a completed duplicate that PayloadRelay identifies. The downstream delivery can still be pending.
`451`	A pause or maintenance, an authentication `temperror`, a rate limit or a quota limit, a billing block, or a different temporary service problem. The sender controls the retry time and the retry duration.
`550`	An unknown, disabled, or wrong-type recipient, usually rejected at `RCPT TO`, a malformed email or body, a permanent authentication error, or a policy rejection.
`452` at `RCPT TO`	The SMTP transaction attempted a second valid relay recipient. The first recipient stays accepted.

Send one PayloadRelay relay recipient in each SMTP transaction.

Email relay loop protection#

An email from a direct PayloadRelay email destination, an email failover, or a webhook response-forward email carries the reserved X-PayloadRelay-Loop header. PayloadRelay permits a maximum of three direct email relay hops. At the limit, PayloadRelay does not send the email. It marks the delivery as failed with a reason in the delivery log. A webhook destination, a chat destination, and the other destinations that are not email continue. PayloadRelay removes the reserved header from the payload metadata that a user sees. The loop protection needs no customer configuration.

Multipart parsing#

PayloadRelay extracts the email bodies from the MIME structure with these rules:

Part type	Extraction behavior
`text/plain`	PayloadRelay extracts it as `textBody`. When the message has more than one `text/plain` part, PayloadRelay uses the first part.
`text/html`	PayloadRelay extracts it as `htmlBody`. When the message has more than one `text/html` part, PayloadRelay uses the first part.
Attachments (a different content type, `Content-Disposition: attachment`, or binary)	PayloadRelay removes them. PayloadRelay relays the text content only.
Inline images or embedded files	PayloadRelay removes them.

Deeply nested messages and attachments#

PayloadRelay can ignore a MIME part with very deep nesting. It removes the attachments, and it does not cut them. The message must stay in the PayloadRelay email size limit.

Use cases#

Alert forwarding: send a system alert email to Slack, Discord, or a webhook.
Form submissions by email: accept a structured email and send it to a webhook or a Google Sheet.
An email-to-API bridge: convert a third-party email notification into an API call through a webhook target.
Monitoring dashboards: send the email alerts to one central webhook endpoint.

← Back to documentation