DictDefence

Sometime soonish, I will be doing a serious rewrite of this software. Being that this was my first Python project beyond “toy” size, it's got some serious warts in it. I'm probably going to keep the same basic code layout and reuse a lot of it, but the 0.6 release is going to have a new config file, and generally, a major overhaul. It's time for house-cleaning.

DictDefence is program written in Python to stop dictionary attacks of all sorts. It is based on the idea behind the Script Kiddie Defence Script, but is a much larger, more extensible project. The basic idea behind DictDefence is the automated blocking of Script Kiddies that run dictionary based attacks on your servers. For all you systems administrators out there, I'm sure you all have seen the lines in your logs where it is a long listing of some IP trying to log in with a ton of different usernames. That is where DictDefence steps in. It monitors logfiles or, even better, a FIFO and logs invalid accesses based upon Perl Compatible Regular Expressions (PCREs). Once an IP goes over your defined threshold, that IP is banned using one of (currently) 5 different means, instantly stopping the dictionary attack. This is designed to work on *nix systems only. If someone wants to rewrite it to work on Winblows as well, feel free.

DictDefense currently runs only on *nix operating systems. Within it you have some different choices about what you use for backend storage as well as what you want to use to block traffic from those nasty Script Kiddies.

• MySQL – The ubiquitous database. Widespread availability and about a zillion tools for using it.
• SQLite – Small, simple, portable and easy to use. This is the default database to be used with DictDefence. NOTE: You can't use this with FreeBSD (see the warning below)
• Shelf – This option uses the Python standard library shelve module. Since this is part of the standard library, if you use this, you won't need any outside libraries. This method will work just fine for small implementations, but it is strongly recommended that you use either SQLite or MySQL for high traffic sites.

• Null Routing – Simple and available on everything. DictDefense simply routes the bad IP to 127.0.0.1.
• PF – The Berkeley Packet Filter. Originally for OpenBSD, but is now available on most of the BSD distros. My personal weapon of choice.
• IPTables – If you are running a Linux distribution, chances are you have this running and ready to go.
• Noban – This option lets you take no action when an IP is banned. It's basically a “dummy” ban. This is useful if you turn on the notify option and just want to be alerted when there is a potential dictionary attack taking place.
• CustomBan – This option lets you specify an external command line for the banning and unbanning of IPs

• You can read from as many files, simultaneously as you would like. Preferably, attach to a FIFO (named pipe) and let DictDefence hum away forever. It also has the ability to read from regular log files and will attach to the new files upon log rotation (similar to tail -F). One other option is to create a UDP listener which you can send to using a custom UDP send program or simply send your syslogd or syslog-ng info to it.

If you like what you've heard so far and you want to try it out, go ahead and download it:

Head on over to the Sourceforge project page to download the latest release

You can also grab the latest revisions from Subversion:

• Trunk: https://svn.splitstreams.com:444/dictdefence/trunk
• Branches: https://svn.splitstreams.com:444/dictdefence/branches

• Fixed an issue where multiple emails and database entries could occur on a permanent ban
• Fixed an error in the MySQL db schema for perm bans

• Fixed a bug with the handling of bad IPs

• Fixed a bug where a permanent ban would not correctly ban the ip

• Improved config processing in handling of values
• Added a “customban” option to allow for user defined ban and unban commands
• Added stop methods to the threads and set all threads to daemon
• Added another failure regex to the default dictdefence.regex file
• Changed the names of the default config files to be *.default

• Added bi-directional traffic blocking in pf

• Lowered the timeout period of queue get in the DB class. This affected CPU usage greatly

• Made the notifier email only contain the accesses from the last TimePeriod seconds
• Added the option of using an Exponential Moving Average as the rate limiter
• Created a separate config option for permanent banning
• Added support for multiple notification emails
• Added an option to set the sender address for the notify email
• Removed old requirement for pyDNS in installer
• Added a debugging command line option
• Fixed a bug in the creation of the directory for the db file
• Changed the DB cleanup to run every configured TimePeriod rather than once a minute
• Massive performance increases in the “shelf” db type in both speed and memory usage
• Changed program flow to incorporate a “worker” thread pool which performs the regular expression matching and whitelist testing before handing off to the database thread.

• Added a noban option to the rules selection
• Added the option to read syslog info directly from a UDP socket
• Removed the dependency on the DNS module
• Fixed a bug where permanent bans were being unbanned
• Added a built-in “shelve” based DB option
• Changed the name of the default database file to be more generic (used with the “sqlite” and new “shelf” database options)
• Made the distro only standard library dependent if the “shelf” database type is used

• Fixed a bug in the int2ip() function

• Fixed a bug in the calculation of the integer IP address.
• Fixed a bug in the reporting of previous bans

• Added email notification options on ban
• Fixed a bug with the config file reader
• Fixed a bug with the ip2int() function
• Various items removed for cleanup

<note important> There is one important thing that I have discovered. You cannot use the SQLite database backend on FreeBSD in this package. It will cause python to core dump. This appears to be a problem with threading + sqlite libraries on FreeBSD. This does NOT happen in Linux. </note>

• Available for download at http://sourceforge.net/projects/mysql-python
• Available via Gentoo portage: dev-python/mysql-python
• Available via FreeBSD ports: databases/py-MySQLdb

• Available for download at http://www.initd.org/tracker/pysqlite/wiki/pysqlite
• Available via Gentoo portage: dev-python/pysqlite
• Available via FreeBSD ports: databases/py-pysqlite23

Once you have the above packages installed, you can simply run, as root, the following from the untarred download:

python setup.py install

That will install the executable dictdefence.py and it's library. It will also copy the configuration files to /etc/dictdefence/.

Open up the 3 configuration files in /etc/dictdefence and modify them as you see fit. All 3 files have documentation in them that tells you what options are available to you and how to use them.

All you need to do is make sure the path exists for your database. The default path is /var/db/dictdefence, but you can change that in dictdefence.conf. Assuming you are using the default, just run:

mkdir -p /var/db/dictdefence

For MySQL, you need to create a database and, optionally, a user to access it with. <note>Security monkey says: Never access your MySQL database from a program as the root user. Security monkey always makes a separate user that has access to only that database</note>

To do this, start the mysql client:

mysql -u root -p

You can leave off the -p if your root user does not have a password.

First, you need to create the database:

mysql> CREATE DATABASE dictdef;

Now create the user:

mysql> GRANT ALL ON dictdef.* TO 'dictuser'@'localhost' IDENTIFIED BY 'dictuserpassword';

Obviously, use at least a different password than “dictuserpassword”. Then, that should be it, you should be ready to go in terms of your database. Make sure your user/password/host/database all match up with what is in your dictdefence.conf.

Once you have everything configured, you can just type the following (as root) on the command line:

dictdefence.py -d

That will start DictDefence as a daemon and start it running in the background.

You can drop the following script in /usr/local/etc/rc.d/ and add the following line to /etc/rc.conf to enable DictDefence on startup.

Add this to /etc/rc.conf:

dictdefence_enable="YES"

And the following script, nameddictdefence, in /usr/local/etc/rc.d/ and chmod 555 dictdefence in that directory:

#!/bin/sh
 
# KEYWORD: shutdown
 
. /etc/rc.subr
 
name="dictdefence"
rcvar=${name}_enable
 
load_rc_config $name
 
: ${dictdefence_name="NO"}
: ${dictdefence_flags=""}
 
start_cmd="${name}_start"
stop_cmd="${name}_stop"
 
pidfile="/var/run/${name}.pid"
 
dictdefence_start() {
    /usr/local/bin/dictdefence.py -d ${dictdefence_flags}
    if [ $? = 0 ] ; then
        echo Dictdefence started
    else
        echo There was a problem starting dictdefence
    fi
}
 
dictdefence_stop() {
    kill `cat ${pidfile}`
    echo Dictdefence stopped
}
 
run_rc_command "$1"

Add this conf file to /etc/conf.d/ and name it dictdefence:

# Set command-line opts here
#DD_OPTS="-c /etc/dictdefence/dictdefence.conf"

And add this script to /etc/init.d/ and name it dictdefence and chmod 755 dictdefence in /etc/init.d/:

#!/sbin/runscript
 
depend() {
        need net
}
 
checkconfig() {
        if [ ! -f /etc/dictdefence/dictdefence.conf ] ; then
                eerror "Please create /etc/dictdefence/dictdefence.conf"
                return 1
        fi
        return 0
}
 
start() {
        checkconfig || return $?
 
        ebegin "Starting dictdefence"
    dictdefence.py -d ${DD_OPTS}
        eend $? "Failed to start dictdefence"
}
 
stop() {
        ebegin "Stopping dictdefence"
        start-stop-daemon --stop \
                --pidfile /var/run/dictdefence.pid 
        eend $? "Failed to stop dictdefence"
}

After doing this, if you wish to have DictDefence run on startup, simply run the following as root:

# rc-update add dictdefence default

Add other systems here…

To get the basic help for the usage of DictDefence, just type dictdefence.py -h on the command line.

You can view the “man page” for it by issuing pydoc dictdefence from the command line.

If there is something you can't seem to figure out or you need help extending the functionality, feel free to email me at admin@splitstreams.com

Bug tracking is up and running at https://bugzilla.splitstreams.com. Head over there to register any bugs.

This work is protected by the GPL version 3. Copyright 2007-2008 Jason Deiman.

A copy of the license is included in the distribution.

~~DISCUSSION~~