Great Circle Associates List-Managers
(May 1998)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: AOL: more trouble than they're worth??
From: Chuq Von Rospach <chuqui @ plaidworks . com>
Date: Wed, 27 May 1998 23:23:43 -0700
To: Ken Hooper <bighouse @ type2 . com>, <list-managers @ GreatCircle . COM>
In-reply-to: <v0311070ab1929b37300e @ [206 . 28 . 74 . 76]>
References: <199805280404 . XAA22852 @ quilla . tezcat . com>

> I think it is absurdly small, but AOL never asks me for my opinion. 8) Back
> when AOL was taking up to 48 hours to produce an error message I made it
> clear to my AOLers that I would delete them on the first "mailbox full"
> error.

Why? You're only making more work for yourself, and creating a problem
that doesn't really exist. And honking off your users for no good
reason.

FWIW, when Brad talked about this in the past, he pointed out that
upgrading the user mailbox to be larger has some rather significant
disk issues for AOL. We're talking about hundreds of gigabytes of extra
disk to handle their user base. they have to make some technical
tradeoffs here -- and where on *my* site upping quotes means calling
APS for a new 9 gig disk to plug into the SCSI array, for AOL, it means
building a new computer building and filling it with lots of disks.
Andcomputers. And staffers. And... Those of us who drive the kind of
boat you can haul on a trailer behind the Suburban just can't
understand how different piloting an aircraft carrier is, and AOL is an
aircraft carrier...

> I don't think the poster knows how good he has it, AOL has improved its
> mail service radically.

Yup. they're the least of my problems. I know, for instance, that when
I get a "user unknown" that user really won't come back three days
later. I wish most of teh net were that reliable.

I don't understand why people get so upset about bounce messages that
they're willing to hack apart their subscriber base because of it. If
administrating lists makes you that crazy, you're in the wrong
business. Imagine if a church told its members that when they called
them on the phone to ask about the next rummage sale and got voicemail
instead of a real person, they'd throw them out of the church. it's
shortsighted and silly. the mailing list isn't there to be convenient
for the admin. It's there to be useful to the users.

rather than focus anger and whatever at AOL (which is at best a symptom
of a bigger problem, and that problem is simple: handling bounce mail
is not fun, boring, repetitive, never ends, and can eat up huge amounts
of time -- but that's not AOL's fault. It's teh simple fact that mail
addresses go stale and mail servers ALL OVER the net can be flakey as
all get out, and AOL gets blamed because they tend to be bigger than
anyone else, not because they're worse. That Chris Craft gets swamped
by the wake of the aircraft carrier, but the aircraft carrier isn't the
only guy churning out wakes. and unlike a LOT of sites, AOL works hard
to minimize its wake and not run over the rest of us small boats. end
digression) -- focus on the problem.

In the "money where your mouth is" world, I'll note I run close to 200
lists now, and have a subscriber base of about 100,000 unique addresses
now (just under, but growing at a net of +350-400 a day). I handle all
but a tiny amount of the bounce mail for all of that, with what's an
enhanced manual system (I have a bounce processor in design for a
summer project...). I have bounce mail issues most folks on this list
can't conceive of. I've come back from long weekends to 20 megabytes of
mail to look through. Such fun....

so here's a few hints from the trenches you might find more useful than
burning effigies of strawmen that aren't really to blame in the first
place.

First thing to figure out: bounce mail isn't something to be taken
personally. It's a fact of life. I don't like paying bills, either, but
I don't threaten my credit card companies because they send me bills...

My philosophy is pretty simple -- I want to get dead email addresses
off my list as fast as reasonable, while minimizing "false positives".
The latter is a lot more important now that you have more and more
spamblocking causing intermittent outages, so you need to make sure a
dead address is really dead. Some sites make this a real adventure.

First thing I do is run all error mail through a series of procmail
rules. These rules do a few things -- one important one is that it
checks the bounce address to see if it's already on my list of
"bounced" addresses. If I've already nuked the address (and nuked
addresses go on a special list for a few days to warn them of this if
they come alive again, so the subscriptions aren't blackholed on
them....), I don't CARE if I get more bounces, so I throw them out.
Second, I rewrite the SUBJECT line of the error mail and encode it with
the email address and the list it bounced on. I then mail it to a
special error mailbox that keeps it out of my normal mail.

(digression again. When I've talked to other list folks about bounce
hassles in the past, many are more frustrated that the bounce mail gets
in the way of their "real" mail than anything else. So use multiple
mailboxes and/or mail filtering to keep it out of sight until you want
to deal with it. don't get mad, get crafty.)

I download it all, and filter mail into special error mailboxes. right
now, I use five (one for digests, one for non-digests, and three for
large mail lists I process separate from the other mail. They're split
this way because each set is handled consistently within itself and I
don't need to do as much thinking trying to keep things straight. The
filtering does it for me...). Eudora lets me sort them by subject
(actually, I sort by subject, sub-sorted by date), and since I encoded
the Subject line, I can tell at a glance what addresses are bouncing
and how often. It's then a matter of determining which ones are "dead",
unsubscribing them, and deleting that set of mail from the folder, and
moving on.

when I'm done, I delete old mail -- I keep a rolling record of the
previous four days. Generally speaking, for a mail address to be "dead"
I want it to be bouncing for three days in a row. So I resort the
folder by date, delete the stuff older than four days, and forget about
it.  Andsince I've put the information I need to make decisions in the
subject line, I don't have to OPEN mail and read it. I don't care WHY
it bounces, only that it is -- and that it does so over a period of
time. I have very, very few false positives, but I don't have huge
amounts of mail bouncing, either (well, in comparison to what happens
when I stop processing stuff for a week or so...).

I generally have ~2000 or so bounce messages in my queues at any time.
I process stuff daily (although I'm going to try three times a week
soon. I think it'll be fine), in an hour to 90 minutes. About half of
that time is spent handling the stuff my mail processing can't parse,
which is actually about 20% of the bounces.

Oh, one other thing my procmail stuff does -- if there are bouncing
addresses that I can't track down (usa.net is horrible about this,
since they don't encode ANY useful identifying info before forwarding
stuff), my procmail scripts throw THOSe bounces out, too. I do my best
to find the right address to unsuscribe, but if sites won't cooperate,
I don't worry about it too much (and I don't waste my time going to
admins on a case by case basis -- not worth my time). Instead, every
few months I do an address validation probe, and dump these bogus
bounces out during that phase all at once.

The trick, folks, is not to get upset because sites send bounce
messages. It's to make bounce processing less onerous. It ain't AOL's
fault here -- they're noticable because they're big, but they're one of
THE MOST stable and reliable e-mail sites out there. Period. I'd say
15-25% of my 100,000 addresses are AOL addresses -- and they're the
least of my email problems.

If you're like me, you hate cleaning the bathroom -- but that doesn't
mean you kick someone out of the house because they used it and forgot
to wipe the sink after washing their hands.... Cleaning the bathroom is
necessary, so either find ways to make it faster and less annoying, or
hire a maid (or both!). but don't get honked at the users of the
bathroom... After all, that's why it's there, no?

--
Chuq Von Rospach (Hockey fan? <http://www.plaidworks.com/hockey/>)
Apple Mail List Gnome (mailto:chuq @
 apple .
 com)
Plaidworks Consulting (mailto:chuqui @
 plaidworks .
 com)
<http://www.plaidworks.com/> + <http://www.lists.apple.com/>


References:
Indexed By Date Previous: Re: AOL: more trouble than they're worth??
From: Ken Hooper <bighouse @ type2 . com>
Next: error returned
From: keath <keath @ pyrotechnics . org>
Indexed By Thread Previous: Re: AOL: more trouble than they're worth??
From: Ken Hooper <bighouse @ type2 . com>
Next: Re: AOL: more trouble than they're worth??
From: dattier @ wwa . com (David W. Tamkin)

Google
 
Search Internet Search www.greatcircle.com