Tags:
create new tag
, view all tags
FightingSpam lists and sometimes describes ways of fighting spam I "found". Then, a few weeks ago, I came up with what might be my own proposal for fighting spam.

Disclaimers re "my": I cover these disclaimers a few times, on this page, other pages on WikiLearn, and other of my writings, either public or private (and I should move these to a "definitive" WikiLearn page to cover the subject thoroughly, once and for all, but) — to hit the high spots briefly, I make no claim that these are necessarily my original ideas (some of them might be, but that's not very important, especially at the moment), and further, if I can find some other proposal that incorporates the "critical factors" of "my" proposal, I will (probably) be happy to support that proposal instead of pushing "my" proposal.

To state my proposal in the simple terms (thanks to one of the guys on the LVLUG list for pointing out that my proposal was not very clear — I'm not at my email machine right now and don't recall his name at the moment):

  • I propose to run a program on my computer which deletes spam from my email account on my ISP before downloading it.

  • I want to encourage my ISP (and eventually all ISPs) to do the same thing (for at least readily identifiable spam) before he accepts email from his upstream SMTP relays.

  • More details are necessary, like what is readily identifiable spam, should there be bounce messages (yes, at least initially), and how to deal with something that might be identified as spam but that I do want to receive.

Clearly, the first part of the proposal is totally under my control, and can be done without anybody else's cooperation. The second part requires my ISPs cooperation, and then the cooperation of many other ISPs. The advantage is that it does not require any kind of top down impetus (like law, fiat, or change to Internet standards), it can be a grass roots movement.

See:

Contents

Notes

<indiscriminantly pasting most of what I deleted from FightingSpam followed by another checkpoint save (there are a few points that might be more appropriate back in FightingSpam on my "working" floppy>

  • If every user ran something like this, bandwidth requirements for the ISP would be reduced (not overall "to the Internet" bandwidth, but bandwidth / machine / modem load to his customers).
  • If every ISP ran a similar program that deleted emails meeting (or failing to meet) certain criteria before he "downloaded" them (really, before he accepted the relay via SMTP) from the next upstream ISP (or carrier), bandwidth requirements for the whole Internet could be reduced. (Really, I probably shouldn't focus on bandwidth requirements, as I don't think email is the major user of Internet bandwidth, I should really say, the spam problem would be reduced for everyone.
  • Oops, I probably am leaving out one piece of the puzzle, IIRC, in some cases, a single email comes to my ISP to be distributed by my ISP to multiple "customers" — if that email is "acceptable" by any of his customers, the ISP must download that email, thus not reducing bandwidth requirements, but potentially increasing CPU utilization and bandwidth requirements to do the "vetting". Hmm, but my first thought is that spam is spam (I'll need to expound on that some more), if it is unacceptable by any (or a majority) of customer(s), it should be rejected for all.

Aside, brainstorming re the above: I'm almost tempted to say that my ISP should feel free to delete any email to me over a certain size (100 KB) unless it's from a special group of whitelisted correspondents. Thus, anybody who wants to send me something over 100 KB must write a short email to me first, and convince me to add them to my whitelist (possibly on a "one time" basis, possibly on a "permanent" basis.)

Hmm, yes, I think I could live with that. What does that do for the ISP? Hmm, if everybody has a different size limit and a different white list, he will need to do a lot of processing before determining whether to accept a downloaded email. EAch customer having a different whitelist makes sense (seems to be a requirement), but I think (hope) that a common agreement could be reached on the size limit. (Currently I'd seriously consider 100 KB as the size limit, as my most annoying problem is the Swen (A?) virus, which is around 106 KB (IIUC). I'd seriously consider something smaller, in the hopes of avoiding HTML mail. Hmm, and would it make any sense for the ISP to support a few limits, i.e., 50 KB, 100 KB, 250 KB, 1 MB?

Hmm, two more things:

  • If my ISP runs a "well known" / "respected" virus checker on "upstream" emails, I'd be content to let him reject upstream mail based on "positives" (even if there were occasional false positives). (Maybe, at least initially, each non-downloaded email deserves a bounce message, saying something like "destination rejected this email because:" (and then one (or more) of) "over 100 KB", "contains HTML", and "not whitelisted by recipient", or whatever.)
  • Darn, I forgot, or was it the bounce message? Ohh, or did I already mention it — each SMTP relay in the chain could/should adapt the same "strategy", so that eventually spam is filtered out at the first SMTP relay that receives it.

Hmm, I don't think I'd mind a bit if my ISP ran two SMTP servers, one with rules like I'm describing, and one with the old traditional rules, and gave me the option of having my current email address being handled by whichever I preferred. I'm sure that, at first, this would be an additional burden on the ISP, but I'm equally sure that over time, more and more people would move from the "old" rule SMTP server to the new rule SMTP server. (Part of the driver for this suggestion is so that people can keep their existing email addresses. They might also opt to have an email address (different) on each mail server, if their ISP wanted to support that.

Hmm, how do I pursue this farther. Well, first I should see if a similar suggestion is being discussed (so first I have to define the critical essence of this proposal so I can decide that some other similar proposal is "acceptable" to me — I'll try that below), then either help push that suggestion, or start exposing this suggestion for more people to consider. One thing I can do is write to someone at the right level at my ISP, and say that I would not object (in fact I would encourage) them to implement something along these lines, and if I lose some email because of it, I will not hold them responsible, or something along those lines).

What are the critical features of this proposal:

(Aside, looks like I've garbaged up this page quite thoroughly — probably need to move this whole discussion to a different page.)

Critical:

  • I run a program like kshowmail (or SwenDeleter) to avoid downloading (i.e., delete at my ISP) emails over 100 KB, and possibly those with HTML or a hint of a virus, psosibly with a bounce message. Then I (or somebody) might refine the script to "whitelist" some senders. Comment: No need for anybody else to do anything, totally under my control.
  • Get my ISP to do something similar to his upstream relays — requires that:
    • A significant portion of his (my ISP's) customers accept the same approach (a willingness to have all emails over 100 KB deleted unless the sender is specifically whitelisted).
    • __
  • __
  • __

Optional:

  • Bounce message: my first thought is that, at least until it (rejecting emails over 100 KB) becomes a common thing on the Internet, a bounce message should be sent. (With instructions along the lines of "if you think the recipient wants this email over 100 KB, you must send him a smaller message and ask him to whitelist you.)
  • Similar to above, give my ISP permission to handle my email address via a new (second) MTA (Mail Transfer Agent) following the new rules I'm discussing. Seek his permission to contact his other customers to suggest that they give the ISP similar permissions (both the first (handle all mail by these rules), and second (move my address to a second server with these new rules).
  • __

One final (perhaps controversial) point, as long as this is sort of a grass roots movement, driven by the email "customer" (i.e., recipient, not the sender), we should (IMHO) be able to avoid any legal issues related to freedom of speech (like the problems with the national do not call registry) or any (new?) laws passed to restrict this kind of approach — as long as it is me, as a customer (and a majority of my fellow customers), saying please don't forward any email to me over 100 KB (rather than some law or "Internet rule" saying emails over 100 KB will not be forwarded to the recipient), we should be able to steer clear of legal problems. (But, since anybody can sue (bring suit) for any reason, there might still be a need to be prepared to defent the approach legally.)

More later, maybe, time for a break.

<text from an email I sent to Chilitech (for possible cut and paste use here>

I may have a misunderstanding, but:

Your note says: "We check every incoming message against several blacklists. ... The blacklist then stops any incoming SPAM from those identified on the list."

I can imagine a few ways of accomplishing this:

  • You "download' the email (i.e., your email SMTP server accepts mail from
your upstream relay), then your software looks at the email on the server and deletes any spam.

  • You download only the headers (one at a time, or as a batch), then, if
you (your software) can determine from the header that it is spam, at the very least you don't download it (and/or refuse delivery) and you may actually delete it from your upstream relay.

I may be misusing and confusing some terms here -- there is software that does things like this at the user level, for example SvenDeleter (Linux only, AFAIK) can run from one of your user's machines, looks at that user's inbox on your email (pop3) server, and if it finds signs of the Sven virus, can actually delete that email on your server, thus never downloading the email (other than the headers). This can be fully automatic or with a review by the user, and can be set up to delete mail based on other criteria.

It is perfectly possible to do the same thing from your email servers point of view. Perhaps your server would simply refuse delivery of the email, which, IIUC, would generate a bounce message. (A good thing.)

I am trying to suggest that ISP's adopt such an approach. I would be willing to let my ISP delete any mail addressed to me that met (or failed to meet) certain criteria:

  • Containing a virus
  • Over a certain size (unless the sender is specifically whitelisted by me)
  • Containing HTML wink

I recognize there are other issues, like emails that come to your server (in an "envelope") addressed to several of your customers, and have started trying to discuss some of the details on http://TWiki.org/cgi-bin/view/Wikilearn/FightingSpamProposal -- it's pretty rough at this point in time.

One of your users thought you were doing something along these lines (deleting mail before accepting it).

regards, Randy Kramer

> >> We check every incoming message against several
> >> blacklists. Blacklists contain the identities of known SPAMMERS. The
> >> blacklist then stops any incoming SPAM from those identified on the
> >> list.

Another email to/from matt at chilitech.com (responding, but off point, to the one in which he made it very clear (thanks Matt!) that the blacklists prevented spam from getting to their server, but spamassassin deleted mail after it had arrived at their server):

Matt,

PS: I forgot a followup question (below):

> On Wednesday 29 October 2003 06:01 pm, mhoppes@chilitechPLEASENOSPAM.net wrote:

> B) spamassassin actually takes the message context/text/words/etc into
> context when it decides if it is dealing with spam or not. The headers
> are rather meaningless.. though they do add to the score. If
> spamassassin has gotten ahold of a piece of spam it is because it was not
> stopped by our first line of defense (blacklists).. so at this point it
> is probably not from a well known spam house.. therefore spamassassin
> must look at each piece of mail to determine if it is spam or not, based
> on the context of th ee-mail, coloured text, wording of sentances, etc...
> it's actually a very inteligent program.

Would it be advantageous (to you, the ISP) if you had a tool like SPAMASSASSIN which did look only at headers and rejected email. Or if your customers gave you permission to not accept messages of the following types (i.e., mail like this does not get into the mailserver, it is rejected during the SMTP handshake):

  • messages over say 100 KB (unless on your customer's whitelist)
  • messages containing HTML
  • ???

Would it further help the entire Internet if you could give similar permission to your upstream relay?

I recognize it would get more complicated, with that upstream relay needing to keep track of who does and who doesn't accept HTML, or who is on who's whitelist, but maybe a "natural evolution" to this is to eventually have something analogous to a DNS system by email recipient -- when mail tries to enter the system, this alternate DNS system is consulted to see if that customer is willing to accept the type of mail (HTML, over 100 KB, fails SPAMASSASSIN test), and if not, the email is rejected at that point.

Two things about that:

  • If the customer gives you permission to do that, I think we can avoid any constitutional challenge re, for example, free speech.
  • In the case of messages over 100 KB, if the customer has a whitelist, maybe the mail is sent through the system anyway to be rejected closer to the recipient (like by you, as his (the recipient's) ISP).

And yes, I recognize that this alternate "DNS" system would be much larger than the current system, due to having entries for each email address, instead of just for each domain. Maybe it's not at all practical, but, on the other hand:

  • processor, memory, and bandwidth all keep growing
  • I'm fairly sure the processor, memory, and bandwidth to support such a system would be less than required to transmit all that email

regards, Randy Kramer

Contributors

  • () RandyKramer - 27 Oct 2003
  • If you edit this page: add your name here; move this to the next line; and if you've used a comment marker (your initials in parenthesis), include it before your WikiName.

Revision Comment

  • %DATE% —

Page Ratings

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r3 - 2003-10-30 - RandyKramer
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding WikiLearn? WebBottomBar">Send feedback
See TWiki's New Look