Mail Mangler

Overview

My webmail does not offer any automated parsing or processing abilities, other than conditional forwards. So, i decided to write (and host) Mail Mangler.

Mail Mangler is a simple passthrough system which takes in emails sent to a special address, performs parsing depending on instructions, and forwards the email in its altered form to a specified destination.

Mail Mangler uses MPL, or Mangler Parsing Language. It is designed to be sent directly as the destination email address to the Mail Mangler Mail Server. This allows users to modify parsing scripts without having to interact with annoying rule processing and parsing interfaces.

Most scripts will probably fit within the 80 character limitation for email addresses. However, MPL has an extensible macro language which can be used to invoke stored parser scripts and calls to other libraries.

Mangler Parsing Language

The parsing script starts with an authentication value, contains parsing instructions in the middle, and ends with a destination value which is mapped to a specific email address.

Authentication ensures that we are not running stray spam messages as arbitrary MPL or acting as an open relay. Destination values are used to both limit the amount of characters needed for a script, as well as a deterent for wanting to "hack" or "scan" authentication codes. This limits any "attackers" to a potentially useless and obscured list of destination email addresses.

That being said, MPL is far from secure, as someone could potentially discover an MPL script in a forwarding rule (or being relayed off an SMTP server they have access to). MPL scripts should be limited to things of little importantance (ie: twitter bots, etc)

Here is an example email address:

A0000|FTranscript|W2|SPlay#message|Z00|@nwrs.net

Each MPL statement is delimited with a pipe symbol. The first character in the statement specifies the type of action or function, and the remaining are arguments.

The parser is linear, and all mail mangler actions are destructive. In other words, finding text with the "B" command will change the start position to this text, deleting all previous parts of the message. "S" will "hide" everything after the search result's first position.

MPL works this way to make it easy to isolate specific pieces of text within an email.

However, sometimes it is nice to concatenate multiple pieces of text within an email or perhaps start fresh from the beginning of the email again. MPL permits the user to "commit" a parse to memory, and either continue onward with the previously hidden message text, or even reload a fresh copy into the parsing buffer again.

This example script allows the user to send speech-to-text tweets using their voice and a telephone:

A0000|BTranscript|W2|SPlay#message|Z00|@nwrs.net

What we will do is configure google voice or gmail to forward new voicemail transcription notifications to Mail Mangler. MPL will parse out the important text and send it off to a twitter email gateway.

Here's an example of part of the body Google Voice will send us:

Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

You've got new voicemail from  (206) 555-1212
Time: 11:58 AM (Aug 17, 2009)
Transcript: Jerry, it's a beautiful day.
Play message:  

Lets break down the parsing instructions to make it tweet the transcript with "A0000|BTranscript|W2|SPlay#message|Z00|@nwrs.net":

A0000 is the user authentication credentials.
BTranscript tells the parser to set the start position of the message to "Transcript", and to include the text we were searching for.
W2 tells the parser to advance to the second word (deleting "Transcript"). We could also use F instead of B to avoid this step.
SPlay#message tells the parser that the text we want stops before "Play Message" (the hash or # mark indicates a space)
Z00 tells the parser to forward the email to the pre-programmed destination "00"

Result: "Jerry, it's a beautiful day." send to a twitter email address stored as "00"

Command Reference

Function	Description	Arguments	Example
A	Authentication statement	Authentication key	A0000
B	Find a string, set that to the start of the message, include the search term	string	BTranscript
C	Number of characters to delete from start of message	number	C12
F	Find a string, set the start of the message to after the search term	string	FTranscript
J	Commits the current message to memory, loads another copy of the original message, lets you edit from the beginning again. Message will be concatenated with what is stored in memory and what is currently being edited	none	J
M	Stored parsing macro	id of macro/arg1/arg2/..	M4923/45/G
P	Append printed text	string	PHello#world
Q	Quit without sending if string is found	string	QNo#transcript#available
R	Replace text with other text	optional string, optional string	Rcrap\stuff
R\	Same as above, deletes all text	none	R\
S	Stop the parser before a string, hiding the remainder of message	string	SPlay#message
W	Delete text before word number specified	number	W2
X	Commit change after S, reveal hidden part of message	X
Z	specify destination email address reference number	number	Z45
.	Evaluate a number to decide an action (see note 1)	operator/value/action	GT/300/Q

Note 1. Evaluate requires the user to isolate a numerical or string value and perform one of the following operators: GE, LE, GT, LT, EQ, NZ, NT, PS, NG (greater than equal to, less than equal to, greater than, less than, equal, not zero, not, positive, negative). Actions consist of Q or C (quit or continue). Naturally, strings will only be able to use the EQ and NT operators, and this function can overlap slightly with Q.

Remapped Characters:

Character	What you should use
(space)	#
:	!
&	@

Google Voice Examples

These examples parse out the following text:

Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

You've got new voicemail from  (206) 555-1212
Time: 11:58 AM (Aug 17, 2009)
Transcript: Jerry, it's a beautiful day.
Play message:

Parse out time and then transcript

A0000|FTime|C3|S#(|X|FTranscript|C3|SPlay#message|Z00

This script looks for "Time", removes the colon and space, inserts the time up to the space and the parans. It then stores the current buffer into memory. We continue on the text and parse the transcript between Transcript and Play message. When we use Z to send, it automatically concatenates what we have in memory.

Result: 11:58AM Jerry, it's a beautiful day.

Create a nice short message detailing the transcript to a cell phone

A0000|R\|PCaller#|X|Ffrom##|STime|X|BTranscript|SPlay#message|Z00

This script will clear the buffer, print "Caller", store to memory and unhide the buffer, parse out the telephone number between "from " and "Time", store to memory and unhide the buffer, parse out the transcript text, and send it to the defined destination.

Result: "Caller (206) 555-1212 Jerry, it's a beautiful day."

Your Bill Is Here Examples

We all hate bills, and here are some examples ways of dealing with them. This is just a small clip from an email notification i get about my power bill.

You have a new e-Bill from PSE - Puget Sound Energy  Biller=
Account Number: ******1111  Due Date: 01/01/2034    Amount Due: 395.10 =
Account Balance:   How do I pay this e-Bill?  Note: If you have=
already paid this bill, please disregard this message. It is simply a remi=
nder that you have an e-Bill.   To pay this e-Bill, follow these steps:  1.=

Get a text notification of your power bill amount

A0000|\R|PPSE#Bill#|X|FAmount#Due|W3|S#|Z01|

First we will clear the buffer, print out "PSE Bill ", store into memory and unhide the rest of the buffer, parse out the dollar amount between "Amount Due" and the space. Send it to the email address for your cell phone (01).

Tweet to your roommate publically when your power bill is over $300

A0000|Ft#Due|W3|S#|.\LT\300\Q|R\|P&geek#PSE#is#$|J|FAmount#Due|W3|S#|Z02|

First, we parse out the amount due in the above email. We perform the evaluate function to determine if it is less than 300. If so, the MPL is told to quit. If not, we clear the buffer without committing it, write "@geek PSE is $395.10", and send it to our email to twitter gateway.