Introductiongps, firstly, is an implementation of a greylist policy service for postfix. Greylisting is a concept to reduce the amount of UCE ('spam') by technical means. Tests on production systems show that greylisting is hugely effective against spam. Read more about greylisting on http://www.greylisting.org and http://projects.puremagic.com/greylisting/whitepaper.htmlSecondly, gps takes greylisting one step further starting with version 1.0. Based on the experience of using greylisting in a production environment, gps comes with features that hugely reduce the problems of the original greylisting concept. These improvements make gps' greylisting usable for ISPs and big mail system setups. gps' main features are:
Project Status
InstallationTo build gps from source the following packages are required:
To build gps unpack the source tar ball (not if using SVN) and run configure and make. Since gps is under development you may have to do: |
tar xvfz gps-<version>.tar.gz [OR] tar xvfz gps-<version>.tar.gz cd gps-X.X (or cd release-<version>) make -f Makefile.cvs ./configure make make install
g++ -s -o gps configreader.cpp db.cpp main.cpp read.cpp cfg.cpp dbdefs.cpp wlcacheddb.cpp signals.cpp -ldbi -ldl
Example for mysql (this does not use a password):
# mysql -p
> CREATE DATABASE greylist;
> GRANT ALL ON greylist TO 'greylist' IDENTIFIED BY 'secret';
> BYE
mode=init.Add gps to your master.cf and main.cf files as described in the postfix documentation under greylisting (taken from http://www.postfix.org/SMTPD_POLICY_README.html):
/etc/postfix/master.cf:
policy unix - n n - - spawn
user=nobody argv=/usr/local/bin/gps /usr/local/etc/gps.conf
/etc/postfix/main.cf:
smtpd_recipient_restrictions =
...
reject_unauth_destination
check_policy_service unix:private/policy
...
policy_time_limit = 3600
| -v | enables verbose log messages | |
| configfile | your config file including path |
| Parameter | Possible values or value range (default in bold) |
Description | Depends on | Version |
|---|---|---|---|---|
| mode | normal | init | weak | reverse | Sets the greylisting mode | ||
| weakbytes | 0 - 4 (3) | Number of significant bytes of client IP address | mode=weak(|reverse) | 0.92 |
| dbtype | mysql | sqlite | pgsql | Database type | ||
| db_host | hostname or IP address | Database server | dbtype | |
| db_username | username | Database user name | dbtype | |
| db_password | password | Database password | dbtype | |
| db_dbname | database name | Database name | dbtype | |
| db_port | port number | Database port | dbtype=pgsql | 0.9 |
| db_pgsql_options | Postgres options | Postgres options | dbtype=pgsql | 0.9 |
| db_pgsql_tty | /dev/ttyX (/dev/null) | Postgres logging | dbtype=pgsql | 0.9 |
| db_sqlite_dbdir | path (permissions!) | SQLite Database path | dbtype=sqlite | 0.9 |
| timeout | seconds (3600=1 hour) | Greylisting timeout | (mode=init) | |
| wl_network | off | db | dbcached (off) | Network whitelisting mode | 0.8 | |
| wl_recipient | off | db | dbcached (off) | Recipient whitelisting mode | 0.8 | |
| wl_sender | off | db | dbcached (off) | Sender whitelisting mode | 0.8 | |
| wl_pattern | off | db | dbcached (off) | Pattern matching whitelisting mode | 0.91 | |
mode tells gps in which mode to run. Default is init init Creates any database table(s) that dont exist. Will always return "dunno" but add records to the database. It can therefore be used to gather a sufficient number of records for the database before switching over to the reverse, normal or weak mode which actually perform the greylisting. Note that this causes more SQL queries as it checks whether the greylist tables exist. This should only be used for initialising the database and testing gps. In the 1.x version init mode should only be used for creating the database tables and not to fill the triplet table with data. The triplets that this mode writes to the database are inconsistent with running gps in mode reverse later. This results in certain mail being let through without any checking. If this is the case there are messages about exceptions in the mail log. The solution is to remove all entries that have client addresses beginning with numbers from the triplet table. Something like this will do the job:
> DELETE FROM triplet WHERE client_address RLIKE '1'; > DELETE FROM triplet WHERE client_address RLIKE '2'; ...
reverse Name based greylisting: instead of checking the client IP address of the triplet this resolves the IP address and uses only a significant part of the host name for checking whether a triplet has already been checked. (E.g. Given the IP address 1.2.3.4 resolves to mail-server255.someisp.com gps checks whether a triplet (someisp.com,sender,recipient) already exists. In normal mode it would check (1.2.3.4,sender,recipient) thus rejecting mail if it came from a different machine on the same relay) This mode is useful for ISPs as it reduces the need to whitelist and therefore complaints from users about not getting mail. At the same time it is still effective in stopping spam. In case the name resolution fails this is logged and weak greylisting based on the IP address is done instead.normal Normal greylisting mode: gps checks the triplets (client IP address,sender,recipient) against the database and blocks mail until the timeout is reached for the first mail. Subsequent mails with the same triplet are let through immediately.weak (Starting from version 0.7) Weak greylisting. In brief this means ignore the last byte of client IP addresses. It is useful if you get stuck with sender mail servers that use a block of network addresses on the same subnet to send mail. Using weak mode results in significantly higher cpu load. It is therefore recommended to use a combination of Whitelisting methods (for version 0.x).timeout The greylisting whitepaper suggests a timeout of 3600 seconds (1 hour) before a new triplet of sender, recipient, client address should be allowed through the greylisting system. Reducing the timeout will keep users happy and is still very effective (e.g. 60 seconds). Default is 3600 dbtype sets the database type to use. This must be set to the same name libdbi expects. Currently, libdi-drivers support mysql, pgsql (version 0.9+), sqlite (version 0.9+), msql (?), oracle (?). gps will exit and log a list of available drivers if the specified driver is not installed (comes in handy for checking libdbi installation). Default value is mysql db_<db parameter> This is the list of paramters to be passed on to libdbi to make the database connection. The db parameters depend on the driver. See the example configuration file below and the included gps.conf, gps.sqlite.conf and gps.pgsql.conf for examples of how to use the different database backends. Example db parameters for dbtype=mysql: db_host=localhost db_username=gps db_password=secret db_dbname=greylist
reverse solves the issues with mail relays and thus reduces the need to whitelist.The following whitelisting options are provided by gps. The can be used in any combination. If you try to optimise your configuration bear in mind that the whitelisting tables get processed before the triplets table.
wl_<module>=<mode>
| off | Is the default. This whitelisting module is not used. | |
| db | (version 0.7b+) The whitelisting data is stored in a table with the name of the whitelisting module. If gps is run in mode=init it will check if the table exists and create it if necessary. Setting a module to db makes gps check evry triplet against the whitelisting module's table before checking the main triplets table. Therefore, for every whitelisting modules enabled one more SQL query is generated. | |
| dbcached | (version 0.8+) When gps is started it reads the module's whitelisting table and creates a memory cache of it which it uses to do subsequent lookups. This uses more memory than db and results in longer startup times, but means fewer SQL queries, and is - once initialised - much faster than db. |
wl_network (version 0.7b+) This sets the network whitelisting mode. If it is set to wl_network=db it will check the table network prior to everything else whether the client address network block has been whitelisted. In order to turn it off use wl_network=off. Default off Example of adding a whitelisting entry in mysql
> use greylist;
> insert into network values ('192.168.0.','my home network');
> bye (or CTRL+D)
wl_recipient (version 0.8+) This sets the recipient (or sender) whitelisting mode. If it is set to wl_recipient=db it will check the table recipient prior to everything else whether the recipient address has been whitelisted. In order to turn it off use wl_recipient=off. Default off Example of adding a whitelisting entry in mysql
> use greylist;
> insert into recipient values ('bla@mydomain.com','this user wants his spam');
> bye (or CTRL+D)
wl_pattern (version 0.91+) allows whitelisting based on regular expression matching.The regular expressions in wl_pattern can, theoretically, be used to replace any of the other whitelisting modules. Furthermore, it can be used to implement complex whitelisting rules combining several conditions. Nevertheless, it should only be used if none of the other modules suit the task. It is much slower by itself and also because all its patterns will be tested against any incoming triplet. The other modules use database or string map based lookups. If wl_pattern has to be used this should be done by setting it to wl_pattern=dbcached thus reducing the number of database queries.
gps builds a text that expressions can be matched against for advanced whitelisting solutions. The format of the gps internal representation is:
s=someuser@yahoo.com r=someuser@mydomain.org c=216.145.54.171 h=mrout1.yahoo.com
In the above example the IP address resolves to one of Yahoo's servers. This pattern uses reverse name lookup and matches the example:
> insert into pattern values(".+^h=.*yahoo\.com.+$","yahoo");
> insert into pattern values(".+^r=.*@someorg\.org.+$","someorg want all spam");
^s=user.+^r=myuser@mydomain.+^c=210
^s=user.+^r=myuser@mydomain\.org.+^c=210
org in the example is still required!s=user is at the beginning do not use the leading ^ ^s=sender@example\.com.+$
weakbytes sets the number of significant bytes of the client address in weak greylisting mode. The default is 3.mode=reverse dbtype=mysql db_host=localhost db_username=gps db_password=secret db_dbname=greylist timeout=60 wl_recipient=dbcached wl_network=db wl_sender=off wl_pattern=dbcached
To test gps and your configuration use the following command. Configuration errors will be logged to syslogd (facility mail). Also see Running.
./src/gps -v etc/gps.conf < tests/testinput4.txt
Now wait for the number of seconds specified in timeout and run the same line again. It should return "action=dunno" lines. If it does gps is ready.
> TRUNCATE TABLE `triplet`;
Example configuration files for postgres and SQLite are in the etc/ folder after unpacking gps. The gps.pgsql.conf contains step by step instruction on how to install and configure postgres on debian and how to create the greylist database and user.
mode=init. Nevertheless, it is useable after this.
gps logs its actions to the syslog mail facility. The output from a testrun is shown below:
mail gps[2225]: started (ver.: 0.8 built: Sep 14 2004 18:35:14) mail gps[2225]: reading config: /etc/gps.conf mail gps[2225]: config: prefix: key: mode value=normal mail gps[2225]: config: prefix: db key: host value=localhost mail gps[2225]: config: prefix: db key: username value=greylist mail gps[2225]: config: prefix: db key: password value= mail gps[2225]: config: prefix: db key: dbname value=greylist mail gps[2225]: config: prefix: key: timeout value=60 mail gps[2225]: connecting to DB, using driver mysql mail gps[2225]: setting DB option: dbname to: greylist mail gps[2225]: setting DB option: host to: localhost mail gps[2225]: setting DB option: password to: mail gps[2225]: setting DB option: username to: root mail gps[2225]: connected to DB mail gps[2225]: ok: 'foobar.tld' -> 'barfoo.tld', '1.2.3.4' (3, 152 secs) mail gps[2225]: action=dunno mail gps[2225]: new: 'foo@blabla.org' -> 'blabla@foo.org', '192.168.0.1' mail gps[2225]: action=defer_if_permit Service is unavailable mail gps[2225]: wait: 'foo@blabla.org' -> 'blabla@foo.org', '192.168.0.1' (0, 34 secs) mail gps[2225]: action=defer_if_permit Service is unavailable mail gps[2225]: disconnecting from DB
| sender | the sender's address | |
| recipient | the recipient's address | |
| client_address | the client's address | |
| client_name | the significant part of the resolved client name when run in mode reverse | |
| count | number of times triplet has been passed from postfix | |
| time_difference | interval between now and the record's time |
mail gps[18838]: wl recipient: 'foobar.tld' -> 'bla@mydomain.com', '192.168.0.254': this user wants his spam
mail gps[18452]: wl network: 'foobar.tld' -> 'bla@someorg.org', '192.168.0.254': my home network
gps-maintain.pl [-v] [-delete] -eq|-lt count -age seconds configfile
/usr/local/bin/gps-maintain.pl -delete -eq 0 -age 18000 /usr/local/etc/gps.conf
/usr/local/bin/gps-maintain.pl -delete -age 3110400 /usr/local/etc/gps.conf
reverse: see description under modereverse and fallover to weak if necessaryTriplets table to triplet (this should make it easier to track version change problems) #define OLDWEAK (true)
mysql> explain Triplets; +----------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+--------------+------+-----+---------+-------+ | client_address | varchar(40) | | PRI | | | | recipient | varchar(160) | | PRI | | | | sender | varchar(160) | | PRI | | | | ip64 | decimal(4,0) | | PRI | 0 | | | ip32 | decimal(4,0) | | PRI | 0 | | | ip16 | decimal(4,0) | | PRI | 0 | | | ip8 | decimal(4,0) | | PRI | 0 | | | count | int(11) | | | 0 | | | uts | int(11) | | | 0 | | +----------------+--------------+------+-----+---------+-------+
svn co https://greylist.svn.sourceforge.net/svnroot/greylist/tags/release-1.005 greylist-1.005The current development version is available from subversion on sourceforge
svn co https://greylist.svn.sourceforge.net/svnroot/greylist greylist
Older versions will be available from this site but mainly for archiving reasons. If you are using one of them consider upgrading to the most recent stable version.
1.5.1