Ubuntu Manpage: smtpd-filters — filtering API for the smtpd daemon

NAME

       smtpd-filters — filtering API for the smtpd daemon

DESCRIPTION

       The smtpd(8) daemon provides a Simple Mail Transfer Protocol (SMTP) implementation, which allows ordinary
       machines  to  become  Mail eXchangers (MX).  Some features that are commonly used by MX, such as delivery
       reporting or spam filtering, are outside the scope of SMTP and too complex to fit in smtpd(8).

       Because an MX may need to provide these features, smtpd(8) provides an API to extend its behavior through
       smtpd-filters.

       At runtime, smtpd(8) can report events to smtpd-filters, querying what it should answer to these  events.
       This allows the decision logic to rely on third-party programs.

DESIGN

       smtpd-filters  are  programs  that  run as unique standalone processes, they do not share smtpd(8) memory
       space.  They are executed by smtpd(8) at startup and expected to run in an infinite loop, reading  events
       and  filtering requests from stdin(4), writing responses to stdout(4) and logging to stderr(4).  They are
       not allowed to terminate.

       Because smtpd-filters are standalone programs that communicate with smtpd(8) through fd(4), they may  run
       as different users than smtpd(8) and may be written in any language.  smtpd-filters must not use blocking
       I/O, they must support answering asynchronously to smtpd(8).

REPORT AND FILTER

       The API relies on two streams, report and filter.

       The  report  stream  is a one-way stream which allows smtpd(8) to inform smtpd-filters in real-time about
       events.  Report events do not expect an answer  from  smtpd-filters;  they  are  just  meant  to  provide
       information.   A  filter  should  be  able  to  replicate  the  smtpd(8) state for a session by gathering
       information coming from report events.  No decision is ever taken by the report stream.

       The filter stream is a two-way stream which allows smtpd(8) to query smtpd-filters about what  it  should
       do  with  a session at a given phase.  Filter requests expect an answer from smtpd-filters; smtpd(8) will
       not let the session move forward until then.  A decision must always be taken by the filter stream.

       It is sometimes possible to rely on filter requests to gather information,  but  because  a  response  is
       expected  by  smtpd(8),  this  is  more costly than using report events.  The correct pattern for writing
       filters is to use report events to create a local state for a session, then use filter requests  to  take
       decisions  based  on  this  state.   The only case when using filter requests instead of report events is
       correct is when a decision is required for the filter request and there is no need for  more  information
       than that of the event itself.

PROTOCOL

       The  protocol  consists  of  human-readable  lines  exchanged between smtpd-filters and smtpd(8), through
       fd(4).

       The protocol begins with a handshake.  First, smtpd(8) provides smtpd-filters with general  configuration
       information in the form of key-value lines:

             config|smtpd-version|6.6.1
             config|smtp-session-timeout|300
             config|subsystem|smtp-in
             config|ready

       Then, smtpd-filters register the stream, subsystem and event they want to handle:

             register|report|smtp-in|link-connect
             register|ready

       Finally,  smtpd(8)  emits  report  events  and filter requests, expecting smtpd-filters to respond or not
       depending on the stream:

             report|0.5|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25
             report|0.5|1576147242.200225|smtp-in|link-connect|7641dfb3798eb5bf|mail.openbsd.org|pass|199.185.178.25:31205|45.77.67.80:25
             report|0.5|1576148447.982572|smtp-in|link-connect|7641dfc063102cbd|mail.openbsd.org|pass|199.185.178.25:24786|45.77.67.80:25

       The character “|” may only appear in the last field of a payload, in which case it should be considered a
       regular character and not a separator.  No other field may contain a “|”.

       The list of subsystems and events, as well as the format of requests and responses, are documented in the
       sections below.

CONFIGURATION

       During the initial handshake, smtpd(8) emits a series of configuration keys  and  values.   The  list  is
       meant to be ignored by smtpd-filters that do not require it and consumed gracefully by filters that do.

       There are currently three keys:

             config|smtpd-version|6.6.1
             config|smtp-session-timeout|300
             config|subsystem|smtp-in

       When smtpd(8) has sent all configuration keys, it emits the following line:

             config|ready

REPORT EVENTS

There is currently only one subsystem supported in the API: smtp-in.

Each report event is generated by smtpd(8) as a single line similar to the one below:

The format consists of a protocol prefix containing the stream, the protocol version, the timestamp, the
subsystem, the event and the unique session identifier, separated by “|”:

It is followed by a suffix containing the event-specific parameters, also separated by “|”:

mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

The list of events and event-specific parameters for smtp-in are as follows:

link-connect: rdns fcrdns src dest
This event is generated upon connection.

rdns contains the reverse DNS hostname for the remote end or an empty string if none.

fcrdns contains the string “pass” or “fail” depending on if the remote end validates FCrDNS.

src contains either the IP address and port of the source address, in the format “address:port”,
or the path to a UNIX socket in the format “unix:/path”.

dest holds either the IP address and port of the destination address, in the format
“address:port”, or the path to a UNIX socket in the format “unix:/path”.

link-greeting: hostname
This event is generated upon display of the server banner.

hostname contains the hostname displayed in the banner.

link-identify: method identity
This event is generated upon “HELO” or “EHLO” command from the client.

method contains the string “HELO” or “EHLO” indicating the method used by the client.

identity contains the identity provided by the client.

link-tls: tls-string
This event is generated upon successful negotiation of TLS.

tls-string contains a colon-separated list of TLS properties including the TLS version, the
cipher suite used by the session and the cipher strength in bits.

link-disconnect
This event is generated upon disconnection of the client.

link-auth: result username
This event is generated upon an authentication attempt by the client.

result contains the string “pass”, “fail” or “error” depending on the result of the
authentication attempt.

username contains the username used for the authentication attempt.

tx-reset: [message-id]
This event is generated when a transaction is reset.

If reset took place during a transaction, message-id contains the identifier of the transaction
being reset.

tx-begin: message-id
This event is generated when a transaction is initiated.

message-id contains the identifier for the transaction.

tx-mail: message-id result address
This event is generated when client emits “MAIL FROM”.

message-id contains the identifier for the transaction.

result contains “ok” if the sender was accepted, “permfail” if it was rejected or “tempfail” if
it was rejected for a transient error.

address contains the e-mail address of the sender. The address is normalized and sanitized, the
characters “<” and “>” are removed, along with any parameters to “MAIL FROM”.

tx-rcpt: message-id result address
This event is generated when client emits “RCPT TO”.

message-id contains the identifier for the transaction.

result contains “ok” if the recipient was accepted, “permfail” if it was rejected or “tempfail”
if it was rejected for a transient error.

address contains the e-mail address of the recipient. The address is normalized and sanitized,
the characters “<” and “>” are removed, along with any parameters to “RCPT TO”.

tx-envelope: message-id envelope-id
This event is generated when an envelope is accepted.

envelope-id contains the unique identifier for the envelope.

tx-data: message-id result
This event is generated when client has emitted “DATA”.

message-id contains the unique identifier for the transaction.

result contains “ok” if server accepted the message for processing, “permfail” if it has not been
accepted and “tempfail” if a transient error prevented message processing.

tx-commit: message-id message-size
This event is generated when a transaction has been accepted by the server.

message-id contains the unique identifier for the SMTP transaction.

message-size contains the size of the message submitted in the “DATA” phase of the SMTP
transaction.

tx-rollback: message-id
This event is generated when a transaction has been rejected by the server.

message-id contains the unique identifier for the SMTP transaction.

protocol-client: command
This event is generated for every command submitted by the client. It contains the raw command
as received by the server.

command contains the command emitted by the client to the server.

protocol-server: response
This event is generated for every response emitted by the server. It contains the raw response
as emitted by the server.

response contains the response emitted by the server to the client.

filter-report: filter-kind name message
This event is generated when a filter emits a report.

filter-kind may be either “builtin” or “proc” depending on if the filter is an smtpd(8) builtin
filter or a proc filter implementing the API.

name is the name of the filter that generated the report.

message is a filter-specific message.

filter-response: phase response [param]
This event is generated when a filter responds to a filtering request.

phase contains the phase name for the request. The phases are documented in the next section.

response contains the response of the filter to the request, it is either one of “proceed”,
“report”, “reject”, “disconnect”, “junk or” “rewrite”.

If specified, param is the parameter to the response.

timeout
This event is generated when a timeout happens for a session.

FILTER REQUESTS

There is currently only one subsystem supported in the API: smtp-in.

Filter requests allow smtpd(8) to query smtpd-filters about what to do with a session at a particular
phase. In addition, they allow smtpd-filters to alter the content of a message by adding, modifying, or
suppressing lines of input in a way that is similar to what program like sed(1) or grep(1) would do.

Each filter request is generated by smtpd(8) as a single line similar to the one below. Fields are
separated by the “|” character.

The format consists of a protocol prefix containing the stream, the protocol version, the timestamp, the
subsystem, the filtering phase, the unique session identifier and an opaque token that the filter should
provide in its response:

It is followed by a suffix containing the phase-specific parameters of the filter request, also separated
by “|”:

mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

Unlike with report events, smtpd(8) expects answers from filter requests and will not allow a session to
move forward until the filter has instructed smtpd(8) how to treat it.

For all phases except “data-line”, responses must follow the same construct: a message of type
“filter-result”, followed by the unique session id, the opaque token, a decision and optional decision-
specific parameters:

The possible decisions for a “filter-result” message are documented below.

For the “data-line” phase, smtpd-filters are fed a stream of lines corresponding to the message to
filter, terminated by a single dot:

They are expected to return an output stream similarly terminated by a single dot. A filter may add to,
suppress, modify or echo back the lines it receives. Ultimately, smtpd(8) assumes that the message
consists of the output from smtpd-filters.

Note that filters may be chained, and the lines that are input into a subsequent filter are the lines
that are output from a previous filter.

The response to “data-line” requests use their own construct. A “filter-dataline” prefix, followed by
the unique session identifier, the opaque token and the output line as follows:

The list of events and event-specific parameters for smtp-in are as follows:

connect: rdns fcrdns src dest
This request is emitted after connection, before the banner is displayed.

helo: identity
This request is emitted after the client has emitted “HELO”.

ehlo: identity
This request is emitted after the client has emitted “EHLO”.

starttls: tls-string
This request is emitted after the client has requested “STARTTLS”.

auth: auth
This request is emitted after the client has requested “AUTH”.

mail-from: address
This request is emitted after the client has requested “MAIL FROM”.

rcpt-to: address
This request is emitted after the client has requested “RCPT TO”.

data This request is emitted after the client has requested “DATA”.

data-line: line
This request is emitted for each line of input in the “DATA” phase. The lines are raw dot-
escaped SMTP DATA input, terminated with a single dot.

commit This request is emitted after the final single dot is received.

For every filtering phase, excepted “data-line”, the following decisions may be taken by a filter:

proceed
No action is taken, session or transaction may be passed to the next filter.

junk The session or transaction is marked as spam. smtpd(8) will prepend an “X-Spam” header to the
message.

reject error
The command is rejected with the message error. The message must be a valid SMTP message
including status code, 5xx or 4xx.

Messages starting with a 5xx status result in a permanent failure, those starting with a 4xx
status result in a temporary failure.

Messages starting with a 421 status will result in a client disconnect.

disconnect error
The client is disconnected with the message error. The message must be a valid SMTP message
including status code, 5xx or 4xx.

Messages starting with a 5xx status result in a permanent failure, those starting with a 4xx
status result in a temporary failure.

rewrite parameter
The command parameter is rewritten.

This decision allows a filter to perform a rewrite of client-submitted commands before they are
processed by the SMTP engine. parameter is expected to be a valid SMTP parameter for the
command.

report parameter
Generates a report with parameter for this filter.

HISTORY

       smtpd-filters first appeared in OpenBSD 6.6.

Debian                                            July 7, 2023                                  SMTPD-FILTERS(7)

NAME

DESCRIPTION

DESIGN

REPORT AND FILTER

PROTOCOL

CONFIGURATION

REPORT EVENTS

FILTER REQUESTS

SEE ALSO

HISTORY