Sending Email from FileMaker Using SparkPost

A bit of background

Back in 2014 we published this article about a new service at the time called Mandrill. Mandrill is an email web service that you can use to send thousands of emails per month. The great thing about Mandrill was that although it had paid subscription options, their free offering was perfect for most FileMaker solutions required to send email. Because email was sent through an API, it also avoids many common SMTP issues that developers come across when trying to setup emailing from FileMaker.

Recently however, Mandrill has announced that it is incorporating Mandrill into its Mailchimp service, and it will become a paid add-on to MailChimp accounts.  This change will take effect from April 27th, 2016 and will affect all existing Mandrill users, and any FileMaker integrations that rely on it.

How does this affect me?

If you are not a Mandrill user, there is no effect, and you can read this post at your leisure. If you do use Mandrill in your FileMaker solution you have cause for taking some urgent action. Here is what is changing:

Mandrill users will be required to have a paid monthly MailChimp account

The separate service known as Mandrill will cease to be its own standalone service and will become incorporated into MailChimp by the end of April. To begin this process, MailChimp is requiring you sign up for an account if not already, and link your existing Mandrill account to it.

This in itself is not really a difficult task, sign up is very easy at mailchimp.com and linking your account is easy too. You have until 27th of April to do this. The following is from Mandrill:

The MailChimp team built Mandrill in 2012 as a transactional email tool. It was a startup within MailChimp that functioned as a completely separate product. Mandrill is now becoming an optional add-on to paid MailChimp accounts. This is a strategic change we’re making to close the gap between the two products. MailChimp is designed for marketing, and Mandrill is designed for transactional messages. Each is powerful on its own, and offering Mandrill as an add-on makes MailChimp even more powerful for our small business and e-commerce customers

The real gotcha with the change is that your MailChimp account will be required to be a paid monthly account by 27 April. Any free accounts will no longer be valid once your Mailchimp and Mandrill accounts are merged.

Pricing is set to change and is not yet finalised on the website. The blog article on the subject details that you will pay a single consolidated monthly fee for both Mailchimp and Mandrill, and Mandrill credits will be sold in blocks of 25,000 emails at $20 per month starting price.

Your sending domains will need to be verified

Mandrill allowed you to send emails from any sending domain. A sending domain is the domain name that comes after the @ part of your email address. This made setup easy but it did mean that the service could be abused by spammers and spoofed email addresses acting as senders.

MailChimp will require that all senders verify their domains. This is actually a good thing and not too difficult to do. Normally verifying your sending domain is done by adding a record into either your SPF or DKIM entries in your domain setup. Your domain provider will be able to assist and provide documentation on how you can do this.

One thing to note here that is important is that you'll need your own verified sending domain and you will be unable to use public domains such as Gmail, Hotmail or Yahoo. For users of free email services such as these it may mean you are out of luck.

SparkPost to the Rescue!

This is where SparkPost comes in. SparkPost is a new web service sending emails much like Mandrill, but SparkPost has a free email plan and is a great alternative for those who wish to not pay for Mandrill. SparkPost is recommended by Mandrill as the alternative for those who wish not to move to MailChimp, that's a good endorsement!

Better still, SparkPost offer a Mandrill Migration Guide which will help anyone with the transition.

SparkPost offers a free 100,00 emails a month plan (capped).

They have a great and fully complete API and excellent developer support.

A couple of Points about SparkPost

Sparkpost will require you to verify your sending domain using SPF/DKIM records, or verification through email. This is the same verification that Mandrill is enforcing, so Gmail/Hotmail/Yahoo users will be out of luck, but those with a legitimate sending domain will have no problem with this.

SparkPost can send emails of up to 20mb in size (with attachments) which should pose no issues for most people.

SparkPost also has a very in depth web based dashboard interface for tracking all mail sent, much like the Mandrill dashboard, and a number of features of Sparkpost are very similar to Mandrill.

Even JSON for formulating an API request is very similar to Mandrill so you should have no trouble with the transition.

Migrating to SparkPost

To give some information about the migration from Mandrill to SparkPost, we have used our old Mandrill example file as the basis, and have modified it to work with SparkPost. You can download the old file below.

The new file is also available here. Below we'll outline some of the important differences between the two that you'll need to be aware of when migrating.

Mandrill Demo

SparkPost Demo

Authentication Differences

When authenticating with Mandrill, your API key was passed as part of your JSON request posted to the API endpoint.

With SparkPost however you must first authenticate using your API key by way of setting a custom POST header prior to sending the JSON request. You also need to specify the content type of your request.  SparkPost will use XML by default so you must specify whether your request is to be in JSON or XML.

To do this, we use the BaseElements plugin from Goya once again. This contains functions for setting your custom headers.

BE_HTTP_Set_Custom_Header ( "Authorization" ; <APIKEY> )

BE_HTTP_Set_Custom_Header ( "Content-Type" ; "application/json" )

Using these two functions, you can authenticate and specify JSON.  These can be combined into a single "Let" statement when posting your request, they need not be specified in separate script steps, or as separate variables, just as long as they're defined prior to posting your request.

Request Differences

While the JSON formatting is very similar to Mandrill, SparkPost has slightly different naming for certain parts of the request such as recipients, but overall it is incredibly similar in how a request is formulated. The demo file contains an equivalent example to the Mandrill demo file, in that it is a template for sending an email to a single recipient with attachments and rich text.

The SparkPost API documentation is great and gives you all the information you'll need to customise the request to suit your needs. The key parts of the documentation around sending emails for you will be the "Transmissions" area and "Recipients" areas. The API can do all kinds of things such as manipulation of your settings, profile, recipient lists, webhooks etc, but a lot of that is not required for simply sending email.

Above is a basic template for sending a SparkPost request. There are only minor changes to that of a Mandrill request, mainly around the structuring of recipients into an array.  The attachment code is basically the same.

Parsing the Result Returned

The format of the result returned by the API is different to that of Mandrill and so you will need to modify any functions you use to parse the response.  SparkPost responses come in one of two varieties. The first is an error that will be returned along with an error code and message describing the error. You will normally see this if you have an improperly formed JSON request, or are missing vital information in your request.

If the JSON request is correct and an email is sent, you will receive a response that gives information about the number of recipients sent to successfully, or number that failed, along with the ID of the request.

An example of an error message:

{ "errors": [ { "message": "required field is missing", "description": "content object or template_id required", "code": "1400" } ] }

This is a JSON error object containing an array of errors. Each object within the array contains 3 attributes about the error. "Message" is the name of the error, and "Description" will give you more information about what the problem is. "Code" is an error code you can use to look up more information on the error in the API documentation. If you have multiple errors you will receive this in an array.

For a successful response, the format is:

{ "results": { "total_rejected_recipients": 0, "total_accepted_recipients": 1, "id": "84290312071045307" } }

This is a results object that contains the total number of rejected recipients, total number of accepted, and the SparkPost ID.

In our demo file we have included a very simplistic parsing function that will transform this into a return delimited list for easier FileMaker parsing. You can nominate which attribute to return, or passing an empty parameter to the function will return the full result. This is similar to the original parsing function in the Mandrill example file.

Rules around what is and is not Spam

If you find that your emails are not being delivered by SparkPost, it may be due to the rules around what SparkPost deems as suspicious or spam emails, and these may be different to Mandrill. SparkPost is quite strict around checking emails for potential spam. Subject lines, email content, recipients etc all contribute to whether the email will bounce or be delivered. 

Originally, our demo file had a test email with the subject line "Hello There!".  While this worked fine in Mandrill, SparkPost considered this email spam. We have had to change the subject line for this example file as a result.  We recommend testing sending various emails after you switch to SparkPost to ensure they are successfully delivered, especially if you use preset templates.

Sending Email using the "Insert from URL" script step

In the Mandrill example file, reader and developer Michael Sloper found a means to send requests to Mandrill in a URL format which allows you to use the Insert from URL script step. this was great because it removed the need for a plugin.

We have been unable to find a suitable means to achieve this using SparkPost - to be fair we haven't really tried very hard!  We welcome anyone who can find a suitable means to integrate with SparkPost using the Insert from URL script step!

In Conclusion

SparkPost is our preferred alternative to Mandrill because it offers the same free plans that Mandrill used to offer, and is very easy to switch to. Our Mandrill article was our most successful ever in terms of reader feedback, and we hope that this article prompts a similar level of discussion and sharing of ideas when it comes to SparkPost integration.

Good luck!

An important note on the API Key and Account used in the Demo file

Because SparkPost requires a valid sending domain for each account, the provided example file has been setup with a preset key on a demo SparkPost account we have created. This demo account has been verified to the sending domain "weetbicks.com" and we have setup a test email address "sparkpost@weetbicks.com" to act as the sender.

You may find that your emails will not send if you use the demo file with a different sender email address. If you wish to do this you should create your own SparkPost account, and obtain your own API key and verified sending domain to use in the example file.

Example File

We've included a Demo file to help you get started on your road to SparkPost integration. You can download using the link provided.

Click here to download the Sparkpost example file.

You can also check out our Mandrill example file for comparison below.

Click here to download the Mandrill example file.