Enterprise Network Consulting logo

Spam filters


    I decided to build this page because spam continues to be a growing problem. If you include your real email address when you post to public newsgroups, if you send more than a minimum number of emails or give out your address to more than just a few select people, or if you have your email address published on people-finder services or web pages you know what I mean.

    Programs available:


    Services available:

    What I did before on my Netscape mail program was just manually build filters. I was always adding to this because spammers are always coming up with new wording for their spam partly intentionally to avoid filters and partly to find new ways to say what they're saying in an effort to hook new people. I stopped updating these filters for awhile, however the percentage and total number of spam messages I receive has continued to rise so I have resorted to manually filtering again. The Thunderbird crew says this is redundant when using their automatic Baeysian filters, however I achieve a higher accuracy in dealing with spam in my own filters than the automatic filters in Thunderbird achieve, and since I receive between 7,000 and 10,000 emails total per month I need all the accuracy I can get just to keep from being overwhelmed. My manual spam filters also mark the message as read and move it directly to my trash folder so I don't have to do a thing to it, as opposed to always having to scroll through the spam folder, then manually mark them all as read, select them all and move them all to the trash folder with my delete key. I'm typically seeing at least 150 messages in my spam filter overnight, and sometimes it's twice that these days. Every time I scroll through the messages in my spam folder I'm looking for false positives (messages that aren't spam) and for patterns that I can manually filter on. I generally add at least one new entry to my existing spam filters per day. On only rare occasions now I also add a filter rule to spam that has come into my inbox.
    I discovered how well my manual filters work recently when I was waiting for an email. My Thunderbird automatically checks for and downloads messages every 10 minutes, but I was impatient. I kept clicking my Get Mail button every 30 seconds or so. The interesting thing is, every time I did that I would see Thunderbird say it was downloading 10 or more messages. When it was done, there were no new messages in my inbox or some other folder, nor were there any in my spam folder. They were all spam being filtered by my manual filters! I'd guess I see about one spam message in my inbox each day, so that's about 30 per month. That means my manual filters and the Thunderbird filters combined give me a spam filtering accuracy of better than 99%.
    The way I build my filters was to look at spam messages that get through to my inbox and decide what if anything I can filter on that would be unique to spam and not to regular emails I might receive. Then I copy/pasted that portion to my filters. Feel free to use the examples I've given above, just copy the portion within the quotation marks (but not the quotation marks themselves) and paste them into your own filters.

    Unfortunately, I am always having to add more filters. It takes time away from other things I'd rather be doing. It's really a matter of choice, to spend your time to make your own filters rather than use Mozilla or spend your money paying someone else to write a program or do the filters for me. It's your choice how you deal with spam you receive. I don't have much hope things will get better in the future with spam. Spamming has a very low response  rate, but it's so incredibly cheap to send out literally millions of emails they will keep doing it. Even a response rate of .1% on a million emails sent is still 1,000 responses, for example. There are efforts in the works to make it illegal to spam, and indeed there are some laws on the books in various places for just that. Unfortunately, it's costly to pursue spammers as they intentionally make it difficult to find them because of these laws and because they don't want to receive masses of emails from irate people. I expect there will be progress in tracking down and prosecuting spammers but they will never be eliminated.
    An idea I had was that more incentive is needed for the law enforcement agencies to pursue this more vigorously. My idea was that  if spammers were required to pay for the spam they send out, that would mean they'd have to be registered and thus easily found if they weren't playing by the rules. If they had to pay to send, those who didn't pay at all would be pursued for taxes not paid. I expect this would be an effective incentive: it's all about money, as they say. The spammers would pay for their own enforcement. The down side is, as has been the trend in taxing of other forms, it would spread. I don't think it would take long for taxing of other non-spamming to be created, such as regular emails ordinary people like you and I send, and that would mean I'd be paying money for spam control, this time without a choice.

Update, November, 2007

    The number of spam emails have continued to rise since I last updated this page. Pretty much everyone with an email address is now receiving far more spam than legitimate email, and the increase appears to have no end in sight. Most email servers filter spam before the user even downloads it, and even Outlook Express has built-in filtering. Some places in the world have anti-spam laws in place, and there has actually been some court action. Unfortunately, though, that seems to have at best only incrementally reduced the meteroic rise in the number of spam messages.
    I continue to be amazed by how many spam messages are so obviously misleading. Before I do business with a company, I check them out to convice myself they are legitmate. I would never do business with a company online or otherwise that doesn't at least appear to be completely honest and operating in good faith. I can't imagine anyone else being any different. I will also never deal with a company that mangles their subject line so badly I have a hard time deciphering what it means, or a company that sends out messages that are composed of a paragraph or two of random words strung together and a link at the bottom. I'll never do business with a company who uses my email address as their from address. I think I'm far from unique on these issues either.  The question then is, how can the spammers ever hope to have anyone at all respond? Could it be that they are so focused on getting past the filters arrayed against them that they lose sight of the fact that no one will ever respond unless they appear legitimate? In spite of my inablilty to find the logic in spam there does appear to contintue to be people who respond, people who send in their money only to receive junk products at best, people who keep the spammers rolling in money and rolling out yet more spam.
    I still have my manual filters and update them on occasion to augment the Mozilla (now Thunderbird) Baeysian filtering. Baeysian filtering is supposed to give accuracy at least as good as 95% and I'm inclined to agree it's around there. That still means I have to manually scan through the messages Thunderbird has marked as possible spam in case something was put in there that isn't spam. My process for this is to scroll through, deal with any that aren't spam, then mark all read and delete them from the folder. That way each time I go to check what's in the folder I know it's all new. Tbird does have an option to automaically purge messages after a set period of time, and if your spam volume is low enough that may be fine. For me, though, I get probably 95% spam and 5% legit email: not small volumes so this works best for me right now. Here is an example of my email. The other day my Tbird stopped automatically retreiving emails for a period of perhaps 4 hours. (Tbird had an automatic update window pop up and it stopped everything until I came back to the computer and answered it.) Tbird then started downloading emails. 683 of them. The accumulated backlog of just 4 hours. When it was finished, I had 118 messages in my spam folder (all correctly spam), 8 messages in my inbox and 4 or 5 scattered through various other folders (I have a lot of filters besides my spam filters, that keep me organized by moving new emails from various people or about certain subjecta into various folders). The other 80% or so of the emails were all caught by my manual spam filter rules, marked as read and deleted. I didn't have to do a thing with them.
    Before this incident I didn't realize how well my manual filters still were working and wasn't doing a lot to keep them up. I've since watched a little more closely as I go through my spam folder. I usually look for patterns, something that is obviously never going to be in a legitimate email, and add probably one new manual filter rule per week. Yes, Tbird is already catching them, but fewer messages I have to scroll through each day in the spam folder is still time not spent manually dealing with this electronic plague called spam. Actually, even if I'm breaking even - even if the time spent updating my manual filters equals the time saved by having them - I still feel like I'm Doing Something.
   


If you would like more information on how to configure mail filters, here is a basic set of instructions I found on Verizon's web site:
 http://www.gte.net/announcements/spam.asp