StuffIveLearned.org

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== DictDefence ======

===== What is it? =====
**DictDefence** is program written in Python to stop dictionary attacks of all sorts.  It is based on the idea behind the [[http://mr.uue.org/projects/skds/|Script Kiddie Defence Script]], but is a much larger, more extensible project.  The basic idea behind **DictDefence** is the automated blocking of [[wp>Script Kiddies]] that run dictionary based attacks on your servers.  For all you systems administrators out there, I'm sure you all have seen the lines in your logs where it is a long listing of some IP trying to log in with a ton of different usernames.  That is where **DictDefence** steps in.  It monitors logfiles or, even better, a FIFO and logs invalid accesses based upon Perl Compatible Regular Expressions (PCREs).  Once an IP goes over your defined threshold, that IP is banned using one of (currently) 5 different means, instantly stopping the dictionary attack.  This is designed to work on *nix systems only.  If someone wants to rewrite it to work on Winblows as well, feel free.



==== Config Choices ====
**DictDefense** currently runs only on *nix operating systems.  Within it you have some different choices about what you use for backend storage as well as what you want to use to block traffic from those nasty [[wp>Script Kiddies]].

=== Backends ===
  * MySQL -- The ubiquitous database.  Widespread availability and about a zillion tools for using it.
  * SQLite -- Small, simple, portable and easy to use.  This is the default database to be used with **DictDefence**.  **NOTE**: You can't use this with FreeBSD (see the warning below)
  * Shelf -- This option uses the Python standard library ''shelve'' module.  Since this is part of the standard library, if you use this, you won't need any outside libraries.  This method will work just fine for small implementations, but it is strongly recommended that you use either SQLite or MySQL for high traffic sites.

=== Blocking Methods ===
  * Null Routing -- Simple and available on everything.  **DictDefense** simply routes the bad IP to 127.0.0.1.
  * PF -- The Berkeley Packet Filter.  Originally for OpenBSD, but is now available on most of the BSD distros.  My personal weapon of choice.
  * IPTables -- If you are running a Linux distribution, chances are you have this running and ready to go.
  * Noban -- This option lets you take no action when an IP is banned.  It's basically a "dummy" ban.  This is useful if you turn on the notify option and just want to be alerted when there is a potential dictionary attack taking place.
  * CustomBan -- This option lets you specify an external command line for the banning and unbanning of IPs

=== Readers ===
  * You can read from as many files, simultaneously as you would like.  Preferably, attach to a [[wp>FIFO]] (named pipe) and let **DictDefence** hum away forever.  It also has the ability to read from regular log files and will attach to the new files upon log rotation (similar to ''tail -F'').  One other option is to create a UDP listener which you can send to using a custom UDP send program or simply send your ''syslogd'' or ''syslog-ng'' info to it.

===== Download =====
If you like what you've heard so far and you want to try it out, go ahead and download it:

[[https://sourceforge.net/projects/dictdefence|Head on over to the Sourceforge project page to download the latest release]]

You can also grab the latest revisions from Subversion:\\
  * Trunk: [[https://svn.splitstreams.com:444/dictdefence/trunk]]
  * Branches: [[https://svn.splitstreams.com:444/dictdefence/branches]]


===== Changelog =====


==== Version 0.5.3 ====
  * Fixed an issue where multiple emails and database entries could occur on a permanent ban
  * Fixed an error in the MySQL db schema for perm bans

==== Version 0.5.2 ====
  * Fixed a bug with the handling of bad IPs

==== Version 0.5.1 ====
  * Fixed a bug where a permanent ban would not correctly ban the ip

==== Version 0.5 ====
  * Improved config processing in handling of values
  * Added a "customban" option to allow for user defined ban and unban commands
  * Added stop methods to the threads and set all threads to daemon
  * Added another failure regex to the default dictdefence.regex file
  * Changed the names of the default config files to be *.default

==== Version 0.4.2 ====
  * Added bi-directional traffic blocking in pf

==== Version 0.4.1 ====
  * Lowered the timeout period of queue get in the DB class.  This affected CPU usage greatly

==== Version 0.4 ====
  * Made the notifier email only contain the accesses from the last TimePeriod seconds
  * Added the option of using an Exponential Moving Average as the rate limiter
  * Created a separate config option for permanent banning
  * Added support for multiple notification emails
  * Added an option to set the sender address for the notify email
  * Removed old requirement for pyDNS in installer
  * Added a debugging command line option
  * Fixed a bug in the creation of the directory for the db file
  * Changed the DB cleanup to run every configured TimePeriod rather than once a minute
  * Massive performance increases in the "shelf" db type in both speed and memory usage
  * Changed program flow to incorporate a "worker" thread pool which performs the regular expression matching and whitelist testing before handing off to the database thread.

==== Version 0.3 ====
  * Added a noban option to the rules selection
  * Added the option to read syslog info directly from a UDP socket
  * Removed the dependency on the DNS module
  * Fixed a bug where permanent bans were being unbanned
  * Added a built-in "shelve" based DB option
  * Changed the name of the default database file to be more generic (used with the "sqlite" and new "shelf" database options)
  * Made the distro only standard library dependent if the "shelf" database type is used

==== Version 0.2.2 ====
  * Fixed a bug in the int2ip() function


==== Version 0.2.1 ====
  * Fixed a bug in the calculation of the integer IP address.
  * Fixed a bug in the reporting of previous bans

==== Version 0.2 ====
  * Added email notification options on ban
  * Fixed a bug with the config file reader
  * Fixed a bug with the ip2int() function
  * Various items removed for cleanup

===== Install =====
<note important>
There is one important thing that I have discovered.  You cannot use the SQLite database backend on FreeBSD in this package.  It will cause python to core dump.  This appears to be a problem with threading + sqlite libraries on FreeBSD.  This does NOT happen in Linux.
</note>



==== Required Packages ====  
=== mysql-python (if you use MySQL) ===
  * Available for download at http://sourceforge.net/projects/mysql-python
  * Available via Gentoo portage: dev-python/mysql-python
  * Available via FreeBSD ports: databases/py-MySQLdb
        
=== pysqlite (if you use SQLite) ===
  * Available for download at http://www.initd.org/tracker/pysqlite/wiki/pysqlite
  * Available via Gentoo portage: dev-python/pysqlite
  * Available via FreeBSD ports: databases/py-pysqlite23

==== Installing the files ====
Once you have the above packages installed, you can simply run, as root, 
the following from the untarred download:

<code>python setup.py install</code>

That will install the executable ''dictdefence.py'' and it's library.  It will also copy the configuration files to ''/etc/dictdefence/''.

===== Setup =====
Open up the 3 configuration files in ''/etc/dictdefence'' and modify them as you see fit.  All 3 files have documentation in them that tells you what options are available to you and how to use them.  

==== Setup for SQLite ====
All you need to do is make sure the path exists for your database.  The default path is ''/var/db/dictdefence'', but you can change that in ''dictdefence.conf''.  Assuming you are using the default, just run:

<code>mkdir -p /var/db/dictdefence</code>

==== Setup for MySQL ====
For MySQL, you need to create a database and, optionally, a user to access it with.
<note>Security monkey says: Never access your MySQL database from a program as the root user.  Security monkey always makes a separate user that has access to only that database</note>

To do this, start the mysql client:

<code>mysql -u root -p</code>

You can leave off the ''-p'' if your root user does not have a password.

First, you need to create the database:

<code>mysql> CREATE DATABASE dictdef;</code>

Now create the user:

<code>mysql> GRANT ALL ON dictdef.* TO 'dictuser'@'localhost' IDENTIFIED BY 'dictuserpassword';</code>

Obviously, use at least a different password than "dictuserpassword".  Then, that should be it, you should be ready to go in terms of your database.  Make sure your user/password/host/database all match up with what is in your ''dictdefence.conf''.

==== Starting DictDefence ====
Once you have everything configured, you can just type the following (as root) on the command line:

<code>dictdefence.py -d</code>

That will start **DictDefence** as a daemon and start it running in the background.

===== Startup Scripts =====
==== FreeBSD ====
You can drop the following script in ''/usr/local/etc/rc.d/'' and add the following line to ''/etc/rc.conf'' to enable **DictDefence** on startup.

Add this to ''/etc/rc.conf'':
<code>dictdefence_enable="YES"</code>
And the following script, named''dictdefence'', in ''/usr/local/etc/rc.d/'' and ''chmod 555 dictdefence'' in that directory:
<code bash>
#!/bin/sh

# KEYWORD: shutdown

. /etc/rc.subr

name="dictdefence"
rcvar=${name}_enable

load_rc_config $name

: ${dictdefence_name="NO"}
: ${dictdefence_flags=""}

start_cmd="${name}_start"
stop_cmd="${name}_stop"

pidfile="/var/run/${name}.pid"

dictdefence_start() {
    /usr/local/bin/dictdefence.py -d ${dictdefence_flags}
    if [ $? = 0 ] ; then
        echo Dictdefence started
    else
        echo There was a problem starting dictdefence
    fi
}

dictdefence_stop() {
    kill `cat ${pidfile}`
    echo Dictdefence stopped
}

run_rc_command "$1"
</code>


==== Gentoo Linux ====
Add this conf file to ''/etc/conf.d/'' and name it ''dictdefence'':

<code bash>
# Set command-line opts here
#DD_OPTS="-c /etc/dictdefence/dictdefence.conf"
</code>

And add this script to ''/etc/init.d/'' and name it ''dictdefence'' and ''chmod 755 dictdefence'' in ''/etc/init.d/'':

<code bash>
#!/sbin/runscript

depend() {
        need net
}

checkconfig() {
        if [ ! -f /etc/dictdefence/dictdefence.conf ] ; then
                eerror "Please create /etc/dictdefence/dictdefence.conf"
                return 1
        fi
        return 0
}

start() {
        checkconfig || return $?

        ebegin "Starting dictdefence"
    dictdefence.py -d ${DD_OPTS}
        eend $? "Failed to start dictdefence"
}

stop() {
        ebegin "Stopping dictdefence"
        start-stop-daemon --stop \
                --pidfile /var/run/dictdefence.pid 
        eend $? "Failed to stop dictdefence"
}
</code>

After doing this, if you wish to have **DictDefence** run on startup, simply run the following as root:

<code># rc-update add dictdefence default</code>

==== Other Systems ====
FIXME Add other systems here...

===== Getting Help =====
To get the basic help for the usage of **DictDefence**, just type ''dictdefence.py -h'' on the command line.

You can view the "man page" for it by issuing ''pydoc dictdefence'' from the command line.

If there is something you can't seem to figure out or you need help extending the functionality, feel free to email me at <admin@splitstreams.com>


===== Bug Tracking =====
Bug tracking is up and running at [[https://bugzilla.splitstreams.com]].  Head over there to register any bugs.

===== License =====
This work is protected by the GPL version 3.  Copyright 2007-2008 Jason Deiman.

A copy of the license is included in the distribution.

~~DISCUSSION~~