Search · Index

Popular tags

ad_form (1), ADP (1), ajax (1), aolserver (1), asynchronous (2), bgdelivery (3), bugtracker (1), COMET (1), cvs (4), debian (6), emacs (2), FreeBSD (1), includelets (2), install (2), installation (1), installers (1), javascript (2), libthread (3), linux (3), monitoring (1), nginx (1), oracle (1), osx (2), patches (1), performance (2), postgres (4), pound (1), redhat (1), selenium (1), ssl (1)

No registered users in community xowiki
in last 30 minutes

Contributors

OpenACS.org

Incoming E-Mail

Incoming E-Mail in OpenACS works with the latest version of acs-mail-lite in a general fashion using callbacks. We will first take a look on what needs to be done to get incoming e-mail working and then continue on to see how packages can benefit from this.

The up to date version of this documentation can be found at http://www.cognovis.de/developer/en/incoming_email

Install incoming E-Mail

For the installation instructions we assume the following

hostname: www.yourserver.com
oacs user: service0
linux user: service0
home dir: /home/service0
users mail dir: /home/service0/mail

The service0 user does not have a ".foward" file and is only used for running the OpenACS website.

Install postfix
Install smtp (for postfix)
Install metamail (for acs-mail-lite)
Edit /etc/postfix/main.cf

myhostname=www.yourserver.com
myorigin=$myhostname
inet_interfaces=$myhostname, localhost
mynetworks_style=host
virtual_alias_domains = www.yourserver.com
virtual_maps=regexp:/etc/postfix/virtual
home_mailbox=mail/

Edit /etc/postfix/virtual and add the regular expression to be able to deal with incoming emails.
@www.yourserver.com service0
Edit /etc/postfix/master.cf - uncomment this line so that postfix is able to process incoming emails
smtp inet n - n - - smtpd
Create a mail directory as service0
mkdir /home/service0/mail
Configure Mail Services Lite Parameters

BounceDomain: www.yourserver.com
BounceMailDir: /home/service0/mail
EnvelopePrefix: bounce

The EnvelopePrefix is for bounce e-mails only.

These parameters should be renamed to IncomingDomain and IncomingMaildir and BouncePrefix, to reflect the fact that we are now dealing with more types of incoming e-mails.

Furthermore, setting the IncomingMaildir parameter will be a *CLEAR* indication that incoming email handling is setup. This is useful for other packages to determine if they can rely on incoming e-mail working (e.g. to set the reply-to email to an e-mail address which actually works through a callback if the IncomingMaildir parameter is enabled).
Configure Notifications Parameters

EmailReplyAddressPrefix: notification
EmailQmailQueueScanP: 0

We want to have acs-mail-lite incoming handle the Email Scanning, not each package seperately.
Configure other packages likewise
Call postmap to recompile virtual db:

postmap /etc/postfix/virtual
Restart Postfix

/etc/init.d/postfix restart
Restart OpenACS

Processing incoming e-mail

A sweeper procedure like acs-mail-lite::load_mails should scan the e-mails which are in the Maildir directory on a regular basis and process them (using a parsing procedure). Then it should fire off callbacks. But before it parses the e-mails, it should check if the e-mail came from an auto mailer. Vinod has done this using procmail as follows below, maybe we could get this dragged into TCL code (using regexp or a procmail recipe parser) instead, removing the need for setting up procmail in the first place.

Revised procmail filters:

:0 w
* ^subject:.*Out of Office AutoReply
/dev/null

:0 w
* ^subject:.*Out of Office
/dev/null

:0 w
* ^subject:.*out of the office
/dev/null

:0 w
* ^subject:.*NDN
/dev/null

:0 w
* ^subject:.*[QuickML] Error:
/dev/null

:0 w
* ^subject:.*autoreply
/dev/null

:0 w
* ^from.*mailer.*daemon
/dev/null

To make things granualar a seperate parsing procedure should deal with loading the e-mail into the TCL interpreter and setting variables in an array for further processing.

    ad_proc parse_email {
        -file:required
        -array:required
    } {
        An email is splitted into several parts: headers, bodies and files lists and all headers directly.

        The headers consists of a list with header names as keys and their correponding values. All keys are lower case.
        The bodies consists of a list with two elements: content-type and content.
        The files consists of a list with three elements: content-type, filename and content.

        The array with all the above data is upvared to the caller environment.

} {

}

Processed and put into the array should be:

HEADERS:

message_id
subject
from
to
date
recieved
references
in-reply-to
return-path
.....

X-Headers:

X-Mozilla-Status
X-Virus Scanned
.....

As we do not know which headers are going to be available in the e-mail, we will just set all the headers we can find in the array. The callback implementation would then have to check if a certain header is present or not.

        #get all available headers
        set keys [mime::getheader $mime -names]

        set headers [list]

        # create both the headers array and all headers directly for the email array
        foreach header $keys {
            set value [mime::getheader $mime $header]
            set email([string tolower $header]) $value
            lappend headers [list $header $value]
        }

        set email(headers) $headers

Body parts

An e-mail usually consists of one or more parts. With the advent of complex_send, OpenACS supports sending of multi part e-mails which are needed if you want to send out and e-mail in text/html and text/plain (for old mail readers).

switch [mime::getproperty $part content] {
     "text/plain" {
          lappend bodies [list "text/plain" [mime::getbody $part]]
    }
     "text/html" {
      lappend bodies [list "text/html" [mime::getbody $part]]
   }
}

Handling of incoming files

Due to the fact that we support the tcllib mime functions, getting incoming files to work is pretty easy and straight forward. We just have to look for a part where there exists a "Content-disposition" part. All these parts are file parts. Together with the scanning for the mail body it looks something like this:

        set bodies [list]
        set files [list]

        #now extract all parts (bodies/files) and fill the email array
        foreach part $all_parts {

            # Attachments have a "Content-disposition" part
            # Therefore we filter out if it is an attachment here
            if {[catch {mime::getheader $part Content-disposition}]} {
                switch [mime::getproperty $part content] {
                    "text/plain" {
                        lappend bodies [list "text/plain" [mime::getbody $part]]
                    }
                    "text/html" {
                        lappend bodies [list "text/html" [mime::getbody $part]]
                    }
                }
            } else {
                set encoding [mime::getproperty $part encoding]
                set body [mime::getbody $part -decode]
                set content $body
                set params [mime::getproperty $part params]
                if {[lindex $params 0] == "name"} {
                    set filename [lindex $params 1]
                } else {
                    set filename ""
                }

                # Determine the content_type
                set content_type [mime::getproperty $part content]
                if {$content_type eq "application/octet-stream"} {
                    set content_type [ns_guesstype $filename]
                }

                lappend files [list $content_type $encoding $filename $content]
            }
        }

        set email(bodies) $bodies
        set email(files) $files

Take a note that the files (attachments) are actually stored in the /tmp directory from where they can be processed further. Therefore it is up to the callback to decide if to import the file into OpenACS or not. Once all callbacks have been fired they will have to be deleted again though.

Firing off callbacks

Now that we have the e-mail parsed and have an array with all the information, we can fire off the callbacks. The firing should happen in two stages. The first stage is where we support a syntax like "object_id@yoursite.com". Then incoming e-mail could look up the object_type, and then call the callback implementation specific to this object_type. If object_type = 'content_item', use content_type instead.

ad_proc -public -callback acs_mail_lite::incoming_object_email {

-array:required

-object_id:required

} {

}

callback acs_mail_lite::incoming_object_email -impl $object_type -array email -object_id $object_id

ad_proc -public -callback acs_mail_lite::incoming_object_email -impl user {

-array:required

-object_id:required

} {

Implementation of mail through support for incoming emails

} {

# get a reference to the email array

upvar $array email

# make the bodies an array

template::util::list_of_lists_to_array $email(bodies) email_body

if {[exists_and_not_null email_body(text/html)]} {

set body $email_body(text/html)

} else {

set body $email_body(text/plain)

}

set reply_to_addr "[party::get_by_email $email(from)]@[ad_url]"

acs_mail_lite::complex_send \

-from_addr $from_addr \

-reply_to $reply_to_addr \

-to_addr $to_addr \

-subject $email(subject) \

-body $body \

-single_email \

-send_immediately

}

The object_id based implementations are mainly useful for automatically generated "reply-to" addresses. But with ProjectManager and Contacts it is also handy, because the Project / TaskID is prominently placed on the website. So if you are working on a task and you get an e-mail by your client that is related to the task, just forward the email to "$task_id@server.com" and it will be stored along with the task. HIGHLY useful :).

Obviously you could have implementations for:

forums_forum_id: Start a new topic
forums_message_id: Reply to an existing topic
group_id: Send an e-mail to all group members
pm_project_id: add a comment to a project
pm_task_id: add a comment to a task and store the files in the projects folder (done)

Once the e-mail is dealt with in an object oriented approach we are either done with the message (an object_id was found in the to address) or we need to process it further.

ad_proc -public -callback acs_mail_lite::incoming_email {
-array:required
-package_id
} {
}

array set email {}

parse_email -file $msg -array email
set email(to) [parse_email_address -email $email(to)]
set email(from) [parse_email_address -email $email(from)]

# We execute all callbacks now
callback acs_mail_lite::incoming_email -array email

For this a general callback should exist which can deal with every leftover e-mail and each implementation will check if it wants to deal with this e-mail. How is this check going to happen? As an example, a package could have a prefix, as is the case with bounce e-mails as handled in acs_mail_lite::parse_bounce_address (see below):

ad_proc -public -callback acs_mail_lite::incoming_email -impl acs-mail-lite {
    -array:required
    -package_id:required
} {
    @param array        An array with all headers, files and bodies. To access the array you need to use upvar.
    @param package_id   The package instance that registered the prefix
    @return             nothing
    @error
} {
    upvar $array email

    set to [acs_mail_lite::parse_email_address -email $email(to)]
    ns_log Debug "acs_mail_lite::incoming_email -impl acs-mail-lite called. Recepient $to"

    util_unlist [acs_mail_lite::parse_bounce_address -bounce_address $to] user_id package_id signature

    # If no user_id found or signature invalid, ignore message
# Here we decide not to deal with the message anymore

    if {[empty_string_p $user_id]} {
        if {[empty_string_p $user_id]} {
            ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: No equivalent user found for $to"
        } else {
            ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: Invalid mail signature $signature"
        }
    } else {
        ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: Bounce checking $to, $user_id"

        if { ![acs_mail_lite::bouncing_user_p -user_id $user_id] } {
            ns_log Debug "acs_mail_lite::incoming_email impl acs-mail-lite: Bouncing email from user $user_id"
            # record the bounce in the database
            db_dml record_bounce {}

            if {![db_resultrows]} {
                db_dml insert_bounce {}
            }
        }
    }
}

Alternatively we could just check the whole to address for other things, e.g. if the to address belongs to a group (party)

ad_proc -public -callback acs_mail_lite::incoming_email -impl contacts_group_mail {
    -array:required
    {-package_id ""}
} {
    Implementation of group support for incoming emails

    If the to address matches an address stored with a group then send out the email to all group members

     @author Malte Sussdorff (malte.sussdorff@cognovis.de)
     @creation-date 2005-12-18

     @param array        An array with all headers, files and bodies. To access the array you need to use upvar.
     @return             nothing
     @error
} {

    # get a reference to the email array
    upvar $array email

    # Now run the simplest mailing list of all
    set to_party_id [party::get_by_email -email $email(to)]

    if {[db_string group_p "select 1 from groups where group_id = :to_party_id" -default 0]} {
        # make the bodies an array
        template::util::list_of_lists_to_array $email(bodies) email_body

        if {[exists_and_not_null email_body(text/html)]} {
            set body $email_body(text/html)
        } else {
            set body $email_body(text/plain)
        }

        acs_mail_lite::complex_send \
            -from_addr [lindex $email(from) 0] \
            -to_party_ids [group::get_members -group_id $to_party_id] \
            -subject $email(subject) \
            -body $body \
            -single_email \
            -send_immediately

    }
}

Or check if the to address follows a certain format.

ad_proc -public -callback acs_mail_lite::incoming_email -impl contacts_mail_through {
    -array:required
    {-package_id ""}
} {
    Implementation of mail through support for incoming emails

    You can send an e-amil through the system by sending it to user#target.com@yoursite.com
    The email will be send from your system and if mail tracking is installed the e-mail will be tracked.

    This allows you to go in direct communication with a customer using you standard e-mail program instead of having to go to the website.

    @author Malte Sussdorff (malte.sussdorff@cognovis.de)
    @creation-date 2005-12-18

    @param array        An array with all headers, files and bodies. To access the array you need to use upvar.
    @return             nothing
    @error
} {
    # get a reference to the email array
    upvar $array email

    # Take a look if the email contains an email with a "#"
    set pot_email [lindex [split $email(to) "@"] 0]
    if {[string last "#" $pot_email] > -1} {
       ....
   }
}

Alternatives to this are:

$component_name-bugs@openacs.org (where component_name could be openacs or dotlrn or contacts or whatever), to store a new bug in bug-tracker
username@openacs.org (to do mail-through using the user name, which allows you to hide the actual e-mail of the user whom you are contacting).

Cleanup

Once all callbacks have been fired off, the e-mail needs to be deleted from the Maildir directory and the files which have been extracted need to be deleted as well from the /tmp directory.

References to this Page: es:cookbook Categories: OpenACS Projects (OpenACS.org)