Amazon S3 and CloudFront with WordPress and DreamHost

Amazon web services

Amazon web services

For the last couple of days I have been testing out the Amazon CloudFront service using my blog as a sandbox. My goal was pretty simple: To set up Amazon CloudFront on my blog, enabling static files like images and styles to be distributed for fast access. This post describes my experimentation with it. It is not just straight forward work, and there are different approaches you may choose among. So I thought I should write a short walk through of the steps that I have taken. If you wonder what Amazon CloudFront is, or looking for information on how it could be used in conjunction with WordPress this post can give you an overview and help you getting started.
[% oiopub-banner-2-right %]

Let’s start with Amazon CloudFront and what this actually is. CloudFront is a Content Distribution Network (CDN). The service distributes your data out to multiple datacenter locations, ensuring faster access times. It works together with another service from Amazon S3 (Simple Storage Service). Amazon S3 is a data storage service where your files is stored initially. To be using CloudFront for distribution you need to have a S3 account and store the files there.

Setting up an Amazon Web Services account
As I have blogged about before, I use Amazon S3 already, so it was not necessary for me to create an Amazon Web Services account. If you do not have an Amazon Web Services account head over here to register. After you have registered, you must enable both the Amazon S3 service and the Amazon CloudFront service to your account (just follow the links and sign up).

With an AWS account and enabled both S3 and CloudFront to this account I was ready to start experimenting. Amazon Web Services is just services. This means that they only offers their services via APIs. Uploading a file to your S3 account must be done by a client. If you are a programmer it’s easy to just develop a client, but there are a lot of clients/programs around. An easy and simple client is S3Fox, a Firefox extension that that lets you upload files to Amazon S3 and manage the files’ access control. I use S3 Browser, a great application that handles the administration of my S3 files. Using S3Fox or S3 Browser makes it easy to test the S3 and CloudFront account, making sure it works. Objects (or files) are organized into buckets that can be created, listed, and retrieved using either a REST-style HTTP interface or a SOAP interface.

Update: As Andy says in a comment to this post (thanks Andy!), another client that you could try out is CloudBerry Explorer for Windows or S3HUB for Mac.

The first thing I had to do was to grab the AWS Access Key and the AWS Secret Access Key. Log in to the AWS account and click on ‘Security Credentials’ in the left menu. The first thing I did was to set up the S3 Browser account with this Access Key and Secret Key. I created my bucket and named it ‘static.larre.com‘, then uploaded some files to be sure everything worked:

Testing S3 bucket

Testing S3 bucket

Setting up CloudFront for my WordPress blog
Now I had an Amazon Web Services account, enabled Amazon S3, installed S3 Browser, created a bucket named ‘static.larre.com’, and uploaded files to this bucket. So far so good. Time to concentrate on the Amazon CloudFront and how to integrate it into my blog, that runs WordPress. There are several approaches that can be used. I have tried a couple of different approaches that I’ll try to describe.

Using Dreamhost
Besides hilarious monthly newsletters DreamHost is a great web hosting company. I use DreamHost for hosting of this blog. Last year DreamHost integrated CloudFront into their hosting service. A pretty amusing blog post from DreamHost about it is here. With CloudFront as a “Goodie-service” smoothly integrated in my hosting environment it is really simple to get started. The step is to create a cloudfront domain filling out this form:

CloudFront Domain

CloudFront Domain

What it does is to create a directory where you can put all files and directories that is going to be stored at S3 and distributed by CloudFront. A very simple admin view where you can handle simple tasks like delete, rescan etc is also part of this “goodies” service. When creating the CloudFront Domain, DreamHost creates a distribution for you at Amazon CloudFront. A distribution specifies the location of the original version of your objects. A distribution has a unique CloudFront.net domain name that you can use to reference your objects through the network of edge locations. If you wish, you can also map your own domain name to your distribution. DreamHost adds a CNAME record (in this case it was ‘static’), that has the cloudfront.net address as value. Using a CNAME enables me to use a domain name (in this case static.larre.com) instead of the domain name CloudFront provides. So looking at my own CloudFront distribution I have the following:

  • http://static.larre.com (my own mapped domain to the distribution)
  • http://d20o1nml9my7kd.cloudfront.net (the unique cloudfront.net domain)
  • http://static.larre.com.s3.amazonaws.com (the origin bucket at S3)

One of the most annoying thing about this service, is the lag from I request a rescan until my files are uploaded to S3. During the test period for the last couple of days it has taken everything from 10 minutes to several hours before DreamHost actually did rescan and upload/delete files in my S3 bucket. When DreamHost had finished scanning the directory and uploaded the files to S3, they where accessible. It seems that the only way to upload files is to do it manually within the DreamHost Admin Panel.

Using the service from Dreamhost may be a bit simple and manually copying all the needed files to the specified directory is not a very elegant solution. A more scalable solution is to make some scripts and jobs that pretty much does the work for you. Here is a blog post that describes one solution to make scripts and jobs for copying static files from Dreamhost to Cloudfront.

Adding a WordPress plugin
To work with my CloudFront Domain (http://static.larre.com), I tested a simple WordPress Plugin: ‘My CDN’. This plugin help you offloading javascript, css and theme files to your own CDN network, but only handle url rewriting not actual file transferring. Installing the plugin, activating it, and then adding three input fields under the settings section, was pretty much what I had to do (here is a screenshot of the settings page). One of the problems with this setup and solution, is that I have to manually put files in the specific directory (as described under the DreamHost section) unless I write a bunch of script-code. I think a better solution is to develop or find an already developed plugin that handles the prefixing of static content and also the upload to S3.

I’ve found one plugin: W3 Total Cache developed by Frederick Townes It’s a plugin improving the user experience of your blog by adding page caching, database caching, minify, content delivery network (CDN) functionality and more. Late last year they implemented integration with AWS and CloudFront. I have used the WP Super Cache until now, but it’s necessary to deactivate it before trying to install and activating the W3 Total Cache. Setting up this plugin is straight forward. After installing the plugin and activated it, I rolled over to the settings sections. First of all I had to activate the CDN and chose Amazon CloudFront as the CDN Type. Under the CDN settings I had to enter my Access Key, Secret Key, bucket and CNAME:

cdn settings

cdn settings

The plugin makes it easy to upload whatever static content you have on your WordPress blog to CloudFront. In the settings sections you specify extensions and files for upload to S3. During the testing period (a couple of days) I have found one issue with this plugin. I use WPtouch. It is a mobile theme, that automatically transforms your WordPress blog into a web-application experience when viewed from an iPhone, iPod touch, Android, or BlackBerry Storm touch mobile device. The default settings for W3 Total Cache captured the mobile devices and bypassed WPtouch. I solved it by adding mobile user agents in the corresponding field under the settings for page cache, minify and CDN.

As far as I can see, the DreamHost integration is not the solution for integrating my WordPress blog with Amazon CloudFront. With other needs than to get static WP content out to the edges it may be a nice functionality. But probably you will have to write some scripts and batch jobs working with the CloudFront directory before you get an elegant and effective solution. The W3 Total Cache plugin seems to be an easy and effective tool to use for mye goal. Setting up an AWS account, enabling S3 and CloudFront and adding a CNAME record is what you have to do before working with the plugin. Everything else seems to be achievable within the settings section.

Everything that is referenced with http://static.larre.com is now hosted by Amazon S3 and distributed out by Amazon CloudFront services.

  20 comments for “Amazon S3 and CloudFront with WordPress and DreamHost

  1. Pingback: Weekly linkage

Leave a Reply

Your email address will not be published. Required fields are marked *