Using Amazon SES to send mails from a Ghost blog

As Amazon is blocking all outgoing traffic on port 25 (SMTP) sending mails via Mailgun won’t work. Therefore, I decided to send all mails to subscribers via Amazon Simple Email Service (SES). In this post you can read how I set everything up to make this working.

Photo by Campaign Creators on Unsplash

Why do I want to use Amazon Simple Email Service?

Hello,

Thank you for submitting your request to have the email sending limit removed from your account and/or for an rDNS update.

This account, or those linked to it, have been identified as having at least one of the following:
* A history of violations of the AWS Acceptable Use Policy
* A history of being not consistently in good standing with billing
* Not provided a valid/clear use case to warrant sending mail from EC2

Unfortunately, we are unable to process your request at this time, please consider looking into the Simple Email Service. https://aws.amazon.com/ses/

Regards,
Amazon Web Services

Honestly I didn’t really get the reason because points 1 & 2 are definitely not true as my account is rather new and point 3 seems to be wrong as well, because I used exactly the same description when making the request to get out of the Amazon SES sandbox. But as I wanted to give Amazon SES a try anyways, I didn’t elaborate further on this one.

Another advantage is that I have all my stuff in the AWS ecosystem now, which is very convenient in my opinion. On the other hand, the Ghost integration for Mailgun is a lot better than for Amazon SES. This means there are a lot of things you have to do manually.

Set up Amazon SES

Once you got out of the sandbox, you need to verify your domain. This is necessary to send emails from a domain. I won’t describe how to do this here, because there is a nice tutorial by Amazon again and I don’t see any use in duplicating their content. You can find the tutorial here.

The last step to complete the setup is to get your SMTP credentials. Amazon again provided a nice tutorial for this here.

Use Amazon SES for administrative mails

"mail": { 
"transport": "SMTP",
"options": {
"host": "<SES-HOST>",
"port": 465,
"service": "SES",
"auth": {
"user": "<SES-ACCESS-KEY-ID>",
"pass": "<SES-ACCESS-KEY>"
}
}
}

After completing all of the above described steps sending mails via Amazon SES should work.

NOTE: If you didn’t enable member signup yet, you can read about how to do that here.

Use Amazon SES to inform users about new posts

There are long-term future plans to have bulk email adapters but nothing short term. The integration with Mailgun is not going to be shallow — it will have full stats on deliverability, open rates, click rates, automatic unsubscribe of members that mark newsletters as spam, and so on. Once we have a good idea of how all of that is working then we can look at making a universal API that would allow different adapters to be used but right now we’re focusing our attention so that we can deliver and iterate on the feature quickly. Everything in members is still beta remember ;-)

Therefore, it requires a lot of work to use Amazon SES for this type of emails. Furthermore, I didn’t integrate this functionality in the Ghost core as I want to keep updates as simple as possible. For this reason I decided to implement a semi-automated solution, which means I created a Shell script, that fetches the latest post from my blog and sends an email to all subscribers via Amazon SES. This shell script has to be invoked manually when a new post is created. In the following I’ll describe the necessary steps to complete this task.

Install the AWS Command Line Interface

Create a mail template

{ 
"Template": {
"TemplateName": "MyTemplate",
"SubjectPart": "Greetings, {{name}}!",
"HtmlPart": "<h1>Hello {{name}},</h1><p>Your favorite animal is {{favoriteanimal}}.</p>",
"TextPart": "Dear {{name}},\r\nYour favorite animal is {{favoriteanimal}}."
}
}

With this snippet you can set the following properties:

  • TemplateName: the name of the template
  • SubjectPart: the subject of the mail
  • HtmlPart: the HTML body of the mail
  • TextPart: the text body of the mail if a mail client can't or doesn't display HTML

It is possible to use replacement tags in the subject and body parts. In the above example {{name}} and {{favoriteanimal}} are used as replacement tags and can be replaced with any text when sending the email.

So I created a new HTML file to create the HtmlPart. I was aiming for a very simple mail body, that just shows a short preview of the post and links to it. In the picture below you can see what it looks like, you can find the code for it on GitHub.

Next, I used a Line Break Removal Tool and a JSON Escape/Unescape Tool to stuff the whole JSON into one String, as it is required to create a valid template. I kept the TextPart very simple so the final template looks like this now:

{ 
"Template": {
"TemplateName": "NewPostTemplate",
"SubjectPart": "New Post on ksick.dev: {{postTitle}}",
"HtmlPart": "[html I described above - too long to display here]",
"TextPart": "Hey there\r\n\r\n, A new post is available on ksick.dev:\r\n{{postTitle}}\r\n{{postUrl}}\r\n\r\nAll the best,\r\nKathi from ksick.dev "
}
}

Again, you can find the whole template, which is stored in a file named newPostTemplate.json, on GitHub. Now we got a nice template but Amazon SES doesn't know about it. So let's upload it with the following command:

$ aws ses create-template --cli-input-json file://newPostTemplate.json

Create a SES configuration set to get informed about errors

Write a script to send mails

You can use the command below to send a mail to multiple recipients with the AWS CLI. As you can see, it expects a JSON file, which contains information about the mail itself as an argument. This means in the script this JSON file needs to be created first, before the mail is sent with this command.

aws ses send-bulk-templated-email --cli-input-json file://mybulkemail.json

So let’s take a look at the described JSON file before we dive deeper into the logic of the script. I’ll first show you how it should look like, then I’ll explain its properties.

{
"Source":"Mary Major <mary.major@example.com>",
"Template":"MyTemplate",
"ConfigurationSetName": "ConfigSet",
"Destinations":[
{
"Destination": {
"ToAddresses": [
"anaya.iyengar@example.com"
]
},
"ReplacementTemplateData":"{ \"name\":\"Anaya\", \"favoriteanimal\":\"angelfish\" }"
}
],
"DefaultTemplateData":"{ \"name\":\"friend\", \"favoriteanimal\":\"unknown\" }"
}
  • Source: mail address (and if wanted name) of the sender
  • Template: the name of the template that should be used to create the mail (the one we created above)
  • ConfigurationSetName: the name of the configuration set to use (created in the previous section)
  • Destinations: an array of Destination objects
    -Destination: the recipients
    *ToAddresses: the addresses the mail should be sent to
    *CcAddresses: the addresses the mail should be sent in cc
    *BccAddresses: the addresses the mail should be sent in bcc
    - ReplacementTemplateData: In the template it was possible to define replacement tags (e.g. {{name}}). You can define their replacements for this special destination here.
  • DefaultTemplateData: In the template it was possible to define replacement tags (e.g. {{name}}). You can define their replacements for all destinations here.

This means within the script we need to generate a JSON file containing all of the above described information. The first three, Source, Template and ConfigurationSetName, are rather easy, because they are known constants. The Destinations and DefaultTemplateData properties are going to be more interesting though. So let's see how we can get them.

Get the DefaultTemplateData

To get the necessary data of the latest post we can use the Ghost Content API, to be precise the /content/posts endpoint. To only get title, content and URL of the first post the following parameters should be set:

  • key: an API key you can generate in Ghost's backend. More information can be found here
  • fields: the fields to query, in this case title, url and html for the content
  • limit: how many posts to query, as we only need the latest post, 1 is the value to set

With all that information it is very easy to query the latest post using curl. You can see the code for that below. The keys for title, URL and the content are stored in variables as we will need them again to parse the received data.

title_key="title" 
url_key="url"
content_key="html"
post_data=$(curl -X GET "https://ksick.dev/ghost/api/v3/content/posts/?key=${ghost_api_key}&fields=${title_key},${url_key},${content_key}&limit=1")

Ghost returns a JSON containing an array of posts with the fields we specified when making this request. When it comes to parsing this JSON I found a simply awesome tool to do this: . I can highly recommend checking this one out and playing around with it, because it is just great in my opinion. In the following code snippet you can see how parsing the title, excerpt and URL from the response JSON works. Parsing title and URL is pretty straight forward. For the excerpt it is nearly the same, you just have to parse the first paragraph out of the full content, which can be done with awk. The tr command is used to escape all quotes, because the result will be stored in a JSON again, which I will write about in one of the next sections. The syntax is quite self-explanatory for simple tasks like the one below.

With the above code the JSON for the DefaultTemplateData is stored in a global variable and will be included in the final JSON later on.

Get all Destinations

Again the resulting JSON is stored in a global variable and will be part of the full JSON in the end.

Generate the full JSON template

Finally send the mail

aws ses send-bulk-templated-email --cli-input-json file://$template_file

Full workflow to send a mail to all subscribers

  1. Go to the Ghost admin panel and head to Labs -> Export Content, click Export and download the export file.
  2. Invoke the just created script like this:
./inform_users_about_new_post GHOST_API_KEY PATH_TO_EXPORT_FILE

That’s it, now all subscribers are informed about the new post.

TLDR;

Resources

jq (the awesome tool I used for JSON parsing): https://stedolan.github.io/jq/

Originally published at https://ksick.dev on July 8, 2020.

Creative and detail-oriented software developer. Advanced from mobile to backend development and now getting into full stack solutions.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store