Sending Email from FileMaker Using Mandrill

Introduction

Sending Email from FileMaker is a common feature these days. With the Send Mail script step this has made it even easier as you can either send via an SMTP server or your Mail Client. While good, these methods do have restrictions such as a one attachment limit and no HTML support if using via SMTP server. Also if sending via your Email client then it becomes difficult to capture sent mail in your system and not all mail clients work the same.

Developers have aimed to circumvent these issues by using FileMaker Email plugins capable of HTML and multiple attachments among other things. Plugins are a good alternative but require some setup and configuration of SMTP details, authentication, and are at the mercy of firewalls and other restrictions that may prevent Email from being sent.

Mandrill - What is it and why do I care?

What is Mandrill?

Mandrill is different. It is basically the E-Mail sending side of MailChimp that has been spawned off to become its own site. Because of this, Mandrill also offers a wide range of features that you would expect from an Email campaign management site such as tracking of Email opens and clicks, full statistics on mail sent, bounce management, as well as a raft of other customisable settings.

And why do I care?

You can use Mandrill to send Email via standard SMTP, or via their API. Even if you wish to use it just via SMTP alone it makes sense. The Mandrill SMTP credentials are reliable, secure, and work anywhere (subject to firewall restrictions of course). In addition, you get all the benefits of comprehensive email tracking and settings to help customise the experience.

In this article we will be covering sending Email via Mandrill by using their API. The benefit of this method is the Emails are sent using an HTTP POST request on port 80 so unless you work for a totalitarian company which has blocked the internets then you should be guaranteed that your Emails will send every time.

There is always a catch, what is the catch?

No catch I promise! I may sound like I work for Mandrill but my enthusiasm is based solely on using Mandrill now on 4 separate projects and having great success on all of them. Best of all Mandrill is free (now I have your attention!). With a free account you are given a starting limit of 12,000 Emails per month, with an hourly limit of 250.

Mandrill works using algorithms that fairly allocate Email send capacity based on the type and quantity of Email that you send. If you are sending legitimate Email and not spam then you will begin to notice your daily and hourly limits increase over time - though for most FileMaker solutions these starting limits are more than enough.

Setting up a Mandrill Account

How do I create an account?

Go to http://www.mandrillapp.com and sign up for a new account. Account setup is really easy - all you need is a unique Email address and password. If you intend to use it for multiple solutions then it would be good practice to create a new account per solution - 12,000 emails per day sounds like a lot but you don't want to run out. Accounts are also free so there is no reason not to.

Generating an API Key

All Email you intend to send via the API must be sent with an API Key. The key identifies which account is sending the Email. Once logged into your account select SMTP and API Credentials from the settings menu. Here you can create as many API keys as you wish for various purposes. You will also find the SMTP credentials if you prefer to use those.

Configuring your account for use

Mandrill is packed with configuration settings and tools for managing sent mail. These include things such as click tracking, blacklists, whitelists, custom metadata and webhooks to name a few. We won't go into depth on these here, but we will cover a couple of useful settings found in the Sending Options area.

• Generate plain-text from HTML Emails (and vice versa). This is a useful option for mail client compatibility. Mandrill will automatically generate either HTML or Plaintext from your Email.
• Expose the list of recipients when sending to multiple addresses. You can send an Email to multiple recipients in a single request, however you may not wish for all recipients to know who else got the Email. Use this option to show/hide the full list of recipients from others.
• Send a copy of every Email to this address. A great feature for initial deployment. If you are concerned about Emails not being sent, then you can be CC'd in on everything sent using this option, or use it as a secondary log of everything sent out.
• Forward bounce notifications to this address. If knowing whether an Email reached its destination is important, you can have Mandrill forward a special bounce response to a nominated Email address. We have used this in solutions where we use a plugin to receive these Emails back into FileMaker and process the bounce accordingly

Integration with FileMaker

What do I need to add to FileMaker

From this point on we are going to refer to the Mandrill example file that is included with this article.

Download the example file

There are 3 key pieces of Mandrill information that you need to put into your FileMaker solution:

• The API Key, you can get this from your account.
• The URL to which you will send the API request. Mandrill uses a REST API where you send to various URL's your request depending upon what you are trying to do. For sending Email, the URL is https://mandrillapp.com/api/1.0/messages/send.json
• The request code into which you will place your email information. Best practice is to define the template and substitute placeholders for actual data.

All of the URLs and request formats for the API can be found in the documentation here. We will be sending requests using JSON format, so we use the .json endpoint to the URL - however other formats are available such as XML and PHP.

You can hard-code the above directly into your send scripts, but a better way is to store them in global fields so they can be used system-wide and can be easily modified if required.

Generating an Email send request

The next step is to transform your request template into an actual request that can be sent to the API. You will see in the screenshot above that our request template has some place-holder data in it at various points, eg. SENDERNAME, SUBJECT, and MESSAGE.

How you record that information in the database we will leave entirely up to you. For our example file we have an Email table onto which we will capture this information. For attachments we have a related Attachments table where each record stores an attachment in a container. It is then simply a case of substituting out our placeholders for actual data. To do this we have created a couple of helpful custom functions.

mandrill_createRequest

This custom function takes a number of parameters, each corresponding to something we will substitute into the request. We are also passing the request template as the first parameter, however if you wish it may be easier to include the template within the custom function, particularly if you only intend to use one template for the entire solution. The function is basically a glorified substitute command, but it does a couple of required things:

• It will escape all required characters within the parameters, this is so that the request does not break as a result of prematurely terminated strings caused by a non-escaped characters.
• It will call a helper custom function mandrill_attachmentCode to generate the required request code for attachments.
• Because we are sending all Email as HTML, it will transform the message into HTML code using the GetAsCSS function if it detects it was not passed HTML code already.

EDIT: Thank you to Tim Dietrich, Stephen Dolenski and Michael Gaslowitz who all pointed out we also need to escape other characters in the request so as not to break it!

mandrill_attachmentCode

You don't really need to concern yourself with how this function works, however it is such a cool function function it deserves some mention!

Custom functions can accept a container field as a parameter. However you cannot pass them a list of containers as the List() function is for strings only. We could define 1 parameter per attachment but that is really bad as a) we don't know how many attachments we will have, and b) if we have none or one then we have to define blank for all the other attachment parameters. So how do we process multiple container attachments with a single parameter?

The answer to this came from this Soliant Consulting article written by the very smart Wim Decorte. He noticed that a parameter to a custom function has a dual nature - you can either treat it as a normal value, OR you can treat it as a reference to a value. Basically, if we treat it as a reference then we can use it in conjunction with recursion and the GetNthRecord function to recurse through multiple related attachment containers and process each one. We are making use of the dual nature of the parameter, using it as a both a value and a reference. I think this is one of the coolest untapped features of custom functions!

Attachments are converted into Base64 and along with a file name and mime type are put into the correct request format.

I won't go into any more detail than that, feel free to dig deeper into how it works, and definitely check out Wim's great article on the topic.

Enough with the stalling, send the damn Email already!

This is the easiest part. Now that we have our request generated from the custom function, all that is left to do is send it to the API to send the Email.

The Insert from URL script step may be able to be used to send the request. At the point of writing this, I am still working on determining whether this is possible, and seeking the help of fellow developers in getting this working with this script step. In FileMaker Pro 13, the step does support HTTP POST, but I'm still working on getting the syntax correct for sending. For now, we are going to use the BaseElements plugin to send the request. Thank you to Michael Gaslowitz for pointing out that Insert from URL does support HTTP POST.

The example file comes pre-packaged with the excellent and free BaseElements plugin from Goya Pty Ltd. This should auto-install on startup and prior to sending Email if not already installed. We use this plugin because not only does it have a working mac and windows version but it also has a 64-bit windows version compatible with FileMaker Server 12 and 13 for server-side sending.

Because plugins do not work on iOS you have a couple of options - one is to have FileMaker Server carry out the sending by using the Perform on Server script step. If your solution is not hosted then you can use the native FileMaker Send Mail script step and use the Mandrill SMTP credentials instead of the API. We are also working on getting the Insert from URL step working (see above).

Sending the request is now really easy and is a single function call:

BE_HTTP_POST ( $URL ; $Request )

Where $URL is the API endpoint for sending an email, and $Request is your completed request. Sending of a standard Email without attachments should take less than a second, and a little longer for attachments.

The API Response

After sending the request, the API will send you a response that contains useful information. A typical response will look something like this:

[<br />{"email":"daniel@weetbicks.com",<br /> "status":"sent","_id":<br /> "d54bb019417c491eb01fc8578c4641a5",<br /> "reject_reason":null}<br />]<br />

This is a JSON array of values of the following:

• email. Where the e-mail was sent
• status.This tells you the outcome. If the status is sent or queued then it succeeded. Emails without an attachment will send right away, whereas ones with an attachment will be queued - however they are typically only queued for a couple of seconds so you won't even notice. A status of scheduled means you have told Mandrill to send the message at a nominated date and time (you can define this as part of the request). If the status is rejected or invalid then something went wrong, and you can check the reason for more details
• reject_reason. The reason the Email failed to send if the status is rejected.
• id. This is a unique identifier assigned to the sent Email from Mandrill. This is useful to track the message via the online Mandrill dashboard, and also if you intend to process Email bounces (the bounce message contains the unique ID).

mandrill_parseResponse

We have included this custom function to help you easily obtain a specified parameter from the response. Pass this function the response, along with either the parameter keyword (or associated number) to retrieve the value. This is useful if you wish to run a test condition as to whether the email sent or failed.

So in Conclusion...

We hope this article has given you a good overview of Mandrill, what it is and what its benefits are. Integrating Mandrill with FileMaker is dead simple. It avoids a lot of potential SMTP related issues such as firewall restrictions, Outlook Exchange problems, or simply increasing usage of your own SMTP server. It also contains a wealth of settings and tracking options for marketing/campaign related emails. We haven't even touched upon the comprehensive online dashboard for tracking sent email statistics, but it's all there for you to explore. Have fun!

Example File

Please find attached an example file. This file is provided to help you fully understand what is going on in this article. Note that FileMaker 13 is required to view this example file. The file also contains links to the Mandrill API documentation and website, as well as the custom functions mentioned in the article.

Download Example File

UPDATE:The example file has been updated on 23rd March with the following changes:

• More escaping of request characters within the mandrill_createRequest function
• added a mandrill_escapeData helper custom function to escape merge data in the request
• I have replaced angle bracket for merge data with « and » respectively.