Forum OpenACS Q&A: Improving OACS Bulk Email

Posted by MaineBob OConnor on 07/20/02 09:39 PM

Ben Koot asked "Is there any progress in adding a releable webmail service to ACS?" at the end of a thread over a year old:

openacs.org/bboard/q-and-a-fetch- msg.tcl?msg_id=00024l

I want to share my experience and what I see as important for serious improvements to the OpenACS EMail System.

We have a community of 2000 paying members and 22000 *free* guests. We typically send out 3 messages a week to both groups. This is an openacs 3.x system on Postgresql and Postfix. We use NO pop3 but do provide forwarding (alias) for about 20 team members.

I have customized the tcl scripts that send out messages by going through the pg database. Users who sign up are asked to verify their email by going to a verify webpage by way of URLvars for their user_id and getting a big number using 'epoch' from their registration date. I use the same process in all emails we send for "unsubscribing" ie, instant verification of unsubscribe and a way to instantly subscribe again if they didn't really want to unsubscribe. These webpages do not require cookies so it doesn't matter if they 'tossed their cookies'.

This system works well BUT it is NOT intuitive to the small but growing group of users who don't or can't read!

"Please Do not click Reply. To unsubscribe, click on this link...."

Many Click REPLY and demand "Please unsubscribe me NOW". This group is an obvious pain because I have to look'em up and mark their record....

My Mailing flag in pg keeps track of people who are:

UNVERIFIED: Either they are new or changed their email address. An email is automatically sent to new people and those who change their email address.
VERIFIED: They clicked the link in the email or an admin did it for them.
USER UNSUBSCRIBED: They unsubscribed by clicking the link in any one of our emails.
BOUNCED: The message bounced because their mailbox doesn't exist anymore, or is full or....

Currently I use a timeconsuming process of taking all the bounced email, separated from the ocasional good "reply to" messages and run the bounces into one file where I have scripts to parse out the email addresses and create individual sql statements to look for that email address and mark that record in the user table as a bounce.

AUTOMATION IS NEEDED

I want a system that can accept email replies to a mailbox, and stuff 'em into the database.

Then, have a script read the messages (every few hours) looking for subjects that start "Undeliverable Mail... or "Returned mail: user unknown" or any of the many variations that mailservers use and go and mark the database as Bounce. The script would also mark for messages that say "mailbox full" or "out of office until January 2004" and other autoresponses.... And when the script hasn't seen the message before, forward it for a humanoid to review and addition to the script if appropriate.

I would like to do this with PostFix which is currently working well and not with qmail due to biases....

COMMUNITY BID

I see rough specs appearing here to make this a "Community Bid" (Thanks Malte) and at this point we can commit the small sum of US $100. and more later....

Let us build on this Please... any suggestions?

THANK YOU.
-Bob

PS, for those interested, this .comUnity is www.greatestnetworker.com and to sign up as *MY* guest, go to www.greatestnetworker.com/is/bob/guest

2: Response to Improving OACS Bulk Email (response to 1)

Posted by Ben Koot on 07/20/02 10:54 PM

From what I have been reading on the bboard, and a search on Sourgefource (no tcl based mail functionality)we have 2 options.

1. Integrate an existing mail package from a compting system like php which seems to possibe, as Maltes colleague Venkatesh showed me on Friday during our discussion on documentation at my office.

2. Forget the existing webmail modules loaded on the toolkit because they won't work without a lot of hassle, and start from scratch. Afterall, the current mail system seems to date back the start of ACS. I assume technology has matured since those days.

From an OpenACS promotional point of view I thimk option 1 is not smart. I can't immagine with all the bright folks in the community that have been thinking about this problem for the better part of 3 years ACS core can't handle this. I.e our problem maybe diffrent...

1. Priority.
2. Lack of time.
3. No interest.
4. No challenge.

I would think, given the fact e-mail is far more important to most people than web sites, we should take some action. I am currently working on creating various networking environments where this would be vital. Those folks now use Yahoo. Aslong as I can't offer webmail, I will have a touch job converting these people over to OpenACS. With a bit of luck I will have a modest budget to add to the cause within the next few weeks.

Why not create a communication module, consiting of e-mail, webmail, bulkmail, SMS & chat. Maybe a dumb newbie question, but from an end users point of view it would make the toolkit much more easy to understand.

Just some thoughts on a rainy Saturday evening
Cheers

3: Response to Improving OACS Bulk Email (response to 1)

Posted by David Kuczek on 07/21/02 07:48 AM

The webmail module would need pop3 retrieval, which for some time has to be payed for on yahoo etc. and would need to be stable... I still have the trouble in mind that friends of mine had with gmx (German webmail provider). They incidentally deleted all the messages from a *lot* of customers!!!