python-libmilter

python-libmilter started as a project to maximize the portability of libmilter. I know about, and have used, the excellent CPython implementation by Stuart D. Gathman py-milter. Though I liked this library for it's thin wrapper around the C libmilter library (and thereby speed), I had some issues with getting it to install, having to find and download the appropriate source code for libmilter version installed, etc. This is why I went with a ground up implementation of libmilter in pure Python. I want something I could just “drop in” to a project and have a milter up and running in almost no time, no matter what OS environment I was in.

I also wanted some more choices for getting things done than the default thread approach. This is why I modularized the milter itself to working as a traditional threading milter wherein each client connection gets its own thread, but I also added factories for single thread asynchronous operation and forking. Do note that the threaded version of python-libmilter is constrained by the GIL, whereas py-milter is not since the threading is implemented in C. If you really want to maximize speed, py-milter might be for you, but be aware that I've been able to get excellent performance from this library in production environments with very heavy processing load (evaluating many regular expressions) using the forking factory.

Note that this documentation assumes a good working knowledge of the SMTP protocol as defined in RFC 2822.

You have a couple of options for getting this package. You can head on over to the Github page and download or clone it from there.

You can also use pip or easy_install to install it.

pip install python-libmilter

NOTE: The RPMs are not guaranteed to be the latest versions. Check Github to find out what the latest version is. That said, I will try and keep these up to date.

python-libmilter-1.0.1-1.noarch.rpm
python:python-libmilter-1.0.1-1.src.rpm

Please submit any and all bug reports/feature requests to the Github page.

If you wish to contribute code patches, please use the repository fork and pull request mechanisms of Github.

If you have any other questions, you can always email me directly at admin@splitstreams.com.

You will find that the library file is split up into sections defining the constants for client server options and flags. Note that there are multiple milter versions (2, 3, 4, 6) and not everything is available in earlier versions than 6. See libmilter documentation for specifics on what is available in what versions. I will show where the version 2 compatibility ends so that if you wish to write your app to be compatible with all milter versions, you can do so.

These are flags that are set defining mutations that we wish to make in our app. This first set is compatible with version 2 of the milter protocol.

This next set is available in later milter protocol versions.

There are also some convenience flags.

These are protocol options, and can be set in a couple of different ways. The first way is to simply set them when setting up your factory (TODO: link to the factory section). The other way, at least for the SMFIP_NO* and SMFIP_NR_* options, is to use decorators. That will be explained later. First, we will define the protocol options.

Again, this first section is available to version 2 of the milter protocol.

These other protocol options are defined for new versions of the milter protocol than version 2.

And just like the OPTS, there are some convenience variables for setting all protocol options.

These are what you use to reply to the MTA with what you want it to do currently. The default is to just reply with a CONTINUE (do nothing). You should use these as the return value from your callbacks.

Example:

import libmilter
...
...
def eob(self , cmdDict):
    return libmilter.CONTINUE
...
...

All the rest of the standard constants are defined as well. I've only documented the ones above as these are the only ones directly used.

Here is the list of the overridable callbacks. Any of these can be optionally overridden in a subclass of the MilterProtocol base class.

Note that some of these callbacks include a cmdDict argument. The cmdDict is a dictionary of everything in a fairly raw format that was sent by the MTA for this particular routine. What is contained in it varies depending on your MTA and how it is configured.

As an aside, all this information is also available with a pydoc libmilter.

This gets the connection info:

This gets the HELO string sent by the client

This gets the MAIL FROM envelope address

This gets the RCPT TO envelope address

This gets one header from the email at a time. The “key” is the LHS of the header and the “val” is RHS.

ex.: key=“Subject” , val=“The subject of my email”

This tells you when all the headers have been received

This is called when the client sends DATA

This gets a chunk of the body of the email from the MTA. This will be called many times for a large email.

This signals that the MTA has sent the entire body of the email. This is the callback where you can use modification methods, such as addHeader(), delRcpt(), etc. If you return CONTINUE from this method, it will be the same as an returning ACCEPT.

Here, you can close any open resources.

NOTE: this method is always called when everything is complete.

This is called when an ABORT is received from the MTA.

NOTE: Postfix will send an ABORT at the end of every message.

These are all the methods that can only be used in the eob() callback in your milter to modify the message/senders/recipients themselves. Note that your MTA must support these methods for them to be used. You also must tell the MTA you are going to use these in your application with the appropriate SMFIF_* options passed into the Factory.

This will tell the MTA to add a recipient to the email.

There are essentially 2 different ways you can use this. The most basic way is to just call it with a recipient address:

self.addRcpt('someone@example.com')

That will add an envelope recipient to the message. This can be very useful if you silently want to send certain messages to their another email account, perhaps for spam analysis. To use this, you must set the SMFIF_ADDRCPT option.

The second way is to call this and set the esmtpAdd variable as well. The esmtpAdd variable would be set to optional ESMTP parameters to the add recipient method.
IMPORTANT: If you want to use esmtpAdd to add additional ESMTP parameters, you have to set the SMFIF_ADDRCPT_PAR option instead of (or in addition to) the SMFIF_ADDRCPT option.

This will tell the MTA to delete a recipient from the email

NOTE: The recipient address must be EXACTLY the same as one of the addresses received in the rcpt() callback.

You must have the SMFIF_DELRCPT option set to use this.

This will replace the body of the email with a new body. This is how you would make changes to the body of an email. It is up to you to store the email, modify it and send the entire body back to the MTA with this method.

You must have the SMFIF_CHGBODY option set to use this.

This will add a header to the email in the form: “key: val”

You must have the SMFIF_ADDHDRS option set to use this.

This will change a header in the email. The key should be exectly what was received in header(). If val is empty, the header will be removed. index refers to which header to remove in the case that there are multiple headers with the same key (Received: is one example). Note that index starts at “1”.

You must have the SMFIF_CHGHDRS option set to use this.

This tells the MTA to quarantine the message (put it in the HOLD queue in Postfix). You can optionally set a msg to be potentially logged by the MTA.

You must have the SMFIF_QUARANTINE option set to use this.

Sets the reply that the MTA will use for this message.
The rcode is the 3 digit code to use (ex. 554 or 250).
The xcode is the xcode part of the reply (ex. 5.7.1 or 2.1.0). The msg is the text response.

Example:

self.setReply(555 , '5.7.0' , 'Monkey poo has just been flung at you')

This tells the MTA to change the envelope From address, with optional ESMTP extensions in esmtpAdd.

You must have the SMFIF_CHGFROM option set to use this.

This tells the MTA that we don't want any more of this type of callback.

Unlike the other modifiers, THIS CAN ONLY BE CALLED FROM THE body() callback!!.

You must have the SMFIP_SKIP option set to use this.

When you are building your own milter, you will have to choose a factory to use to run your milter. There are 3 different factories built in, they are:

When creating a factory object, you do so by calling the factory class (AsyncFactory is used as the example here, but all the factories are the same) with the following options.

async = AsyncFactory(socketStr , MyMilter , smfifOptions , tcpListenQueue , sockChmod)

The arguments are as follows:

This factory is somewhat modeled after a Twisted implementation, though with some differences. This will run everything, by default, in a single thread and single process. This means that things must return very quickly (there is a way around this). This is not the recommended factory to use if you are doing anything beyond trivial checks of the message. The advantage to this factory, however, is that it makes sharing of global resources very easy.

The “single thread” model can be circumvented slightly here with the use of a decorator called callInThread that you can use with certain callbacks to send some more intensive processing to another thread, while keeping the main milter thread responsive. Here is a quick example of how to use this:

import libmilter
class MyMilter(libmilter.MilterProtocol):
...
...
    @libmilter.callInThread
    def connect(self , hostname , family , ip , port , cmdDict):
        # Run checks on the IP.  Perhaps this includes RBLs, a local database check, etc.
        if processIP(ip):
            return libmilter.CONTINUE
        else:
            return libmilter.REJECT

NOTE: No “mixin” class needs to be inherited with this factory's use.

This is a factory, wherein each connection from the MTA ends up in it's own thread. This is exactly how the C libmilter works.

This works well, with one caveat, the Python GIL. If you are making a single external call which can take a long time and is not considered blocking (like IO), that call can make your entire daemon unresponsive. An example of this would be a complex regular expression being run that can potentially take a long time to return (I found this out the hard way). If you aren't doing this kind of processing, and especially if you need to share global resources between milter instances, this is probably your best choice. If you need to run complex regular expressions and don't want to block incoming connections from your MTA, use the ForkFactory instead.

NOTE: If you choose use this, your milter protocol class implementation must inherit the ThreadMixin class as well.

Example:

import libmilter
class MyMilter(libmilter.ThreadMixin , libmilter.MilterProtocol):
...

This is a factory that will fork a child process for each incoming connection from the MTA. This will add some more overhead on your machine, but is recommended for the best concurrency. See the description of ThreadFactory as to why this is.

NOTE: If you choose use this, your milter protocol class implementation must inherit the ForkMixin class as well.

Example:

import libmilter
class MyMilter(libmilter.ForkMixin , libmilter.MilterProtocol):
...

See the examples directory in the distribution for a complete test milter example using the ForkFactory.