Ubuntu Manpage: mimedefang-multiplexor - Process pool controller for mail filters.

Provided by: mimedefang_3.3-1build4_amd64

NAME

       mimedefang-multiplexor - Process pool controller for mail filters.

SYNOPSIS

       mimedefang-multiplexor [options]

DESCRIPTION

       mimedefang-multiplexor  manages  a pool of Perl processes for scanning e-mail.  It is designed to work in
       conjunction with mimedefang(8) and mimedefang.pl(8).

       mimedefang-multiplexor opens a UNIX-domain socket and listens for requests for work from mimedefang.   As
       requests  come  in,  mimedefang-multiplexor  creates  Perl  processes  as  needed to scan mail.  The Perl
       processes are not killed when scanning is completed, but continue to run in a loop.  Perl  processes  are
       re-used  for  subsequent  e-mail  messages.   This  eliminates  the large overhead of starting a new Perl
       process for each incoming message.

       To avoid memory leaks, the Perl processes are killed after they have handled some number of scans.

OPTIONS

-U user
Runs the multiplexor as user rather than root. This option is mandatory, and must match the -U
option supplied to mimedefang.

-m minWorkers
The minimum number of Perl processes to keep running at all times. The default is zero.

-x maxWorkers
The maximum number of Perl processes to run simultaneously. If a request comes in and all
processes are busy, a temporary failure is signalled to the SMTP peer. The default is 2.

-r maxRequests
The maximum number of requests a given process handles before it is killed and a replacement
started. The default is 500.

-i idleTime
The idle time in seconds after which to kill of excess Perl processes. That is, if the process is
idle for longer than this time, and there are more than minWorkers running, the process is killed.
Note that this is implemented as a timer which ticks every idleTime seconds; therefore, processes
may be idle for up to twice this time before they are killed. The default for idleTime is 300
seconds.

-V maxLifetime
The maximum lifetime in seconds of a worker before it is killed and a replacement started. The
default is -1, which signifies no maximum lifetime. Note that the lifetime check is done only
when a worker becomes idle after a request, or every time the idle-timeout check is made. On a
very quiet system, workers may live for longer than maxLifetime by as much as idleTime. Note also
that the lifetime is measured not from when the worker started running, but from when it was
assigned its very first request. A completely-idle worker that has never processed any requests
will not be terminated by the maxLifetime setting.

-b busyTime
The longest a Perl process is allowed to spend scanning an e-mail before it is declared hung up
and killed. The default is 120 seconds.

-Z This option specifies that the multiplexor should accept and process "status updates" from busy
workers. Note that this consumes one extra file descriptor per worker, plus a small amount of CPU
time per status update.

-c cmdTime
The timeout for communication between mimedefang-multiplexor and mimedefang, or between
mimedefang-multiplexor and a Perl scanning process. The default is 10 seconds. This timeout
should be kept quite short.

-w waitTime
When mimedefang-multiplexor starts the initial workers, or needs to bring the number of running
workers up to the number defined by the -m option, it does not start all the workers at once,
because this could overload your server. Instead, it starts one worker every waitTime seconds.
The default value for waitTime is 3.

-W waitTime
If you use this option, mimedefang-multiplexor will never activate a worker until waitTime seconds
have elapsed since the last worker activation. This could result in mail being tempfailed if
worker activations do not keep pace with incoming mail. However, it may be preferable to tempfail
mail rather than allow the load on your server to spike up too quickly. The default value for
this option is 0, meaning that mimedefang-multiplexor will start workers as quickly as necessary
to keep up with incoming mail.

-z spooldir
Set the spool directory to spooldir. If this option is omitted, the spool directory defaults to
/var/spool/MIMEDefang.

-s pathName
The UNIX-domain socket on which mimedefang-multiplexor listens for requests. This should be
specified as an absolute pathname. If this option is not supplied, it defaults to mimedefang-
multiplexor.sock under the spool directory.

-a socket
A socket for listening for requests. This is similar to the -s socket, except that a restricted
set of requests are processed. On this socket, the multiplexor will only process requests asking
for status; it will not accept any commands to do scanning or that would consume a worker. See
the SOCKET SPECIFICATION section for the format of socket.

-p fileName
Causes mimedefang-multiplexor to write its process-ID (after becoming a daemon) to the specified
file. The file will be owned by root.

-o fileName
Causes mimedefang-multiplexor to use fileName as a lock file to avoid multiple instances from
running. If you supply -p but not -o, then mimedefang-multiplexor constructs a lock file by
appending ".lock" to the pid file. However, this is less secure than having a root-owned pid file
in a root-owned directory and a lock file writable by the user named by the -U option. (The lock
file must be writable by the -U user.)

-f filter_path
Normally, mimedefang-multiplexor executes a Perl filter script called mimedefang.pl to scan the e-
mail. However, you can have it execute any program you like by specifying the full path to the
program with the -f option. This program must obey the protocol documented in mimedefang-
protocol(7); see that manual page for details.

Note that the -f option does not specify the "filter" to use with mimedefang.pl; instead, it
specifies the program for mimedefang-multiplexor to execute. You almost certainly should not use
this option unless you wish to replace mimedefang.pl with your own program.

-F rules_path
Specifies the path to the filter rules. By default, /etc/mimedefang-filter is used. If you use
the -F option, its value is passed to the underlying Perl filter program using -f.

-l Log certain events, including the output of the Perl workers' standard-error, using syslog.
Normally, the multiplexor does not log much information.

-d Write debugging information about event-handling code in /var/log/mimedefang-event-debug.log.
This is only of use to people debugging mimedefang-multiplexor.

-R kbytes
Limits the resident-set size of the worker filter processes to kbytes kilobytes. This limit is
not supported on all operating systems; it is known to work on Linux.

-M kbytes
Limits the total memory space of worker filter processes to kbytes kilobytes. This limit is
supported on all operating systems which support the setrlimit(2) system call. This should
include most modern UNIX systems.

We recommend that you monitor your worker filter processes and get a feel for how much memory they
use. You should then limit the memory to two or three times the worst-case that you have
observed. This can help mitigate denial-of-service attacks which use complicated MIME messages to
force mimedefang.pl to consume lots of memory.

-h Print usage information and exit.

-t filename
Log statistical information to filename. See the section STATISTICS for more information.

-T Log statistical information using syslog(2). You may use any -t and -T together, in which case
statistical information is logged in a file and using syslog.

-u Flush the statistics file after every write. Normally, mimedefang-multiplexor does not flush the
file; this is the best choice for minimizing disk I/O on a busy mail server. However, if you wish
to watch statistics entries in real-time, you should enable flushing.

-D Do not fork into the background and become a daemon. Instead, stay in the foreground. Useful
mainly for debugging or if you have a supervisory process managing mimedefang-multiplexor.

-q queue_size
Normally, if all workers are busy and mimedefang-multiplexor receives another request, it fails it
with the error "No free workers." However, if you use the -q option, then up to queue_size
requests will be queued. As soon as a worker becomes free, the queued requests will be handed off
in FIFO order. If the queue is full and another request comes in, then the request is failed with
"No free workers".

-Q queue_timeout
Queued requests should not stay on the queue indefinitely. If a queued request cannot be
processed within queue_timeout (default 30) seconds of being placed on the queue, it is failed
with a "Queued request timed out" message. See the section "QUEUEING REQUESTS" for more
discussion.

-O sock
Listen on a notification socket for connections from listeners. mimedefang-multiplexor can inform
external programs of state changes by sending messages over a notification socket. The external
programs connect to this socket and then listen for notifications. See the section SOCKET
SPECIFICATION for the format of sock.

See the mimedefang-notify(7) man page for details of the notification protocol.

-N map_sock
Listen on a map socket for Sendmail SOCKETMAP connections. As of Sendmail 8.13, you can define a
Sendmail map type that talks to a daemon over a socket. mimedefang-multiplexor implements that
protocol; consult the mimedefang-filter(5) man page for detils (see the SOCKET MAPS section).

See the section SOCKET SPECIFICATION for the format of map_sock.

-I backlog
When mimedefang-multiplexor creates a listening socket, it calculates the "backlog" argument to
listen(2) based on the maximum number of workers. However, you can explicitly set this backlog
with the -I option. Setting the backlog to a high value (around 30-50) may help on a very busy
server. If you see mail log messages saying "MXCommand: socket: Connection refused" during busy
periods, then that's an indication you need a higher listen backlog.

-L interval
Log the worker status every interval seconds. This logs a line using syslog; the line looks like
this:

Worker status: Stopped=s Idle=i Busy=b Killed=k Queued=q Msgs=m Activations=a

Here, "Stopped" is the number of non-running workers, "Idle" is the number of idle workers, "Busy"
is the number of busy workers, "Killed" is the number of killed workers yet to be reaped, "Queued"
is the number of queued requests, "Msgs" is the total number of messages processed since the
multiplexor began running, and "Activations" is the number of times a Perl process has been
started since the multiplexor began running.

If you supply an interval of 0 (which is the default), no periodic status logging is performed.
If you supply an interval of less than 5 seconds, it is silently reset to 5 seconds.

-S facility
Specifies the syslog facility for log messages. The default is mail. See openlog(3) for a list
of valid facilities. You can use either the short name ("mail") or long name ("LOG_MAIL") for the
facility name.

-E Specifies that the multiplexor should create an embedded Perl interpreter. This can improve
performance dramatically. But see the section "EMBEDDING PERL" for more information.

-X n Specifies that the multiplexor should initiate a "tick" request every n seconds. This causes your
filter_tick function (if defined) to be called. Note that you have no control over which worker
executes filter_tick. If all workers are busy when a tick occurs, that tick request is skipped
and a warning message is logged.

-P n Specifies that the multiplexor should run n tick requests in parallel. Each tick is run as often
as specified with the -X argument. (If you omit the -P option, then the multiplexor behaves as if
-P 1 had been specified.)

If you run parallel ticks, each tick is assigned an integer identifying its "type". The type
ranges from 0 to n-1. While there may be as many as n tick requests running at a time, only one
tick of each type will be active at any time.

-Y label
Sets the tag used in the multiplexor's syslog messages to label instead of mimedefang-multiplexor.

-G Normally, mimedefang-multiplexor uses a umask of 027 when creating listening sockets. If you
would like the sockets to be readable and writeable by the group as well as the owner, supply the
-G option. This causes the umask to be 007 whenever UNIX-domain sockets are created.

-y n Limits the maximum number of concurrent recipok checks to n on a per-domain basis. The value of n
can range from 0 (in which case no limit is applied) to maxWorkers, where maxWorkers is the
argument to the -x option. If n is outside that range, it is ignored (and no limit is applied.)

The recipok command ultimately invokes the filter_recipient function in your filter. If you are
doing recipient verification against servers that may be slow or unreliable, you can use the -y
option to limit the number of concurrent recipient verifications per domain. That way, if one
domain's server becomes very slow, it won't consume all available workers for recipient
verification. Instead, its RCPT commands will be tempfailed and there will be workers available
to handle RCPT commands for other domains.

SOCKET SPECIFICATION

       The  -a,  -N and -O options takes a socket as an argument.  The format of the socket parameter is similar
       to that of the Sendmail Milter library, and is one of the following:

       /path/to/socket
              A UNIX-domain socket

       inet:portnum
              A TCP socket bound to port portnum, but which accepts connections  only  from  the  IPv4  loopback
              address (127.0.0.1).

       inet_any:portnum
              A  TCP  socket bound to port portnum which will accept connections from any address.  Use inet_any
              with caution!

       inet6:portnum
              A TCP socket bound to port portnum listening on the IPv6 loopback address.

       inet6_any:portnum
              A TCP socket bound to port portnum listening on the IPv6 wildcard address.

QUEUEING REQUESTS

       Normally, if all workers are busy, any additional requests are failed immediately.  However, the  -q  and
       -Q  options  allow  you  to  queue  requests  for  a  short amount of time.  This facility is intended to
       gracefully handle a temporary overload; most of the time, your queue should be empty.

       Because mimedefang checks the number of free workers when a connection is opened and fails the connection
       if there are no free workers, the intent of the queue is to allow  SMTP  transactions  that  are  already
       underway  to  continue  if there is a slight overload.  Any new connections will be failed if all workers
       are busy, but existing connections are allowed to continue.  Queuing requests may improve  throughput  on
       extremely busy servers.

       Note  that  if  you  supply  the -q option to mimedefang, then even new connections are allowed to queue.
       This may improve throughput by keeping the worker utilization higher.

       The -R option to mimedefang can be used to reserve a specified number of workers for connections from the
       loopback address.  Using the -R option has  the  side-effect  of  permitting  new  connections  from  the
       loopback address to queue.

EMBEDDING PERL

       Normally,  when mimedefang-multiplexor activates a worker, it forks and execs mimedefang.pl.  However, if
       the multiplexor was compiled with embedded Perl support, and you supply the -E command-line  option,  the
       multiplexor works like this:

       1      It  creates  an  embedded  Perl interpreter, and sources mimedefang.pl with a special command-line
              argument telling it to read the filter, but not to enter the main loop.

       2      Each time a worker is activated, the multiplexor calls fork()  and  runs  the  mimedefang.pl  main
              loop.  This invokes filter_initialize and then runs the main loop.

       On  some  platforms  (for  example,  Red  Hat  Linux  7.3 with Perl 5.6.1), it is not safe to destroy and
       recreate a Perl interpreter without causing a memory leak.  On those platforms, if you attempt to  reread
       the  filter  file (by sending the multiplexor a HUP signal or reread command), the filter will not be re-
       read, and a message will be logged to syslog.  On those platforms, you must kill and  restart  MIMEDefang
       if you change the filter file.

       On  most  platforms,  however, a filter reread is accomplished by destroying and re-creating the embedded
       interpreter, re-sourcing mimedefang.pl and killing workers as soon as they are idle.

STATISTICS

With the -t option, mimedefang-multiplexor logs certain events to a file. This file can be post-
processed to gather statistics about the multiplexor. You can use it to tune the number of workers you
run, adjust timeouts, and so on.

Each line of the file looks like this:

YYYY/MM/DD:HH:MM:SS timestamp event key=val key=val...

Here, YYYY/MM/DD:HH:MM:SS is the local time of day. Timestamp is the number of seconds since January 1,
1970. Event is the name of an event. The valid events are:

StartWorker
A worker process has been started.

KillWorker
A worker process has been killed.

ReapWorker
A dead worker process has been reaped. It is possible to have a ReapWorker event without a
previous KillWorker event if the worker process terminated abnormally.

StartFilter
A worker process has begun filtering an e-mail message.

EndFilter
A worker process has finished filtering an e-mail message.

The possible keys in the key=value pairs are:

worker=n
The worker involved in the event. Every worker is identified by a small integer.

nworkers=n
The total number of running workers immediately after the event happened.

nbusy=n
The number of busy workers (workers which are processing an e-mail message) immediately after the
event happened.

reason="string"
The reason for a StartWorker or KillWorker event. (Present only for these events.)

numRequests=n
The number of e-mails processed by the worker. Present only for an EndFilter event.

If you send the mimedefang-multiplexor process a SIGHUP signal (kill -1 pid), it closes and reopens the
statistics file. This is useful during log file rotation.

If you send the mimedefang-multiplexor process a SIGINT signal (kill -INT pid), it terminates all active-
but-idle workers. Also, any active-and-busy workers terminate as soon as they finish filtering the
current message. This is useful to force a reread of the filter rules file without stopping and
restarting Sendmail.

If you send the mimedefang-multiplexor process a SIGTERM signal (kill pid), it terminates all workers and
exits immediately.

AUTHOR

       mimedefang-mulitplexor was written by Dianne Skoll <dfs@roaringpenguin.com>.  The mimedefang home page is
       https://www.mimedefang.org/.