Sync Your Magento eCommerce Site To Cloud Files And The Akamai CDN

Filed in Product & Development by Justin Seubert | January 22, 2013 10:00 am

The constant battle a web site administrator must fight is the resource requirements of their site in contrast with the resource requirements of the configuration it is run on. In the end, this will almost always dictate how many user requests can be handled at any point in time. A common tool in the arsenal to fight this battle is to find clever ways to off load these common, or static, requests, off the server and allow it to spend as much time as possible delivering dynamic content as quickly as it can handle. This allows a server to not handle more request than before, but leaves it free to address more than previously it would have.

Most administrators choose a fairly simple process, which is to implement a Content Delivery Network (CDN) solution into their configuration. The premise is simple: upload all static content such as images, css, javascript, html or anything else that does not require dynamic server-side rendering to a cloud CDN provider, and change your site’s code to point to the CDN provider’s servers instead of yours. A lot of the popular content management systems have plugins developed to sync these files for you to a product like Rackspace Cloud Files[1]. From the management end, you then just enable CDN access and the plugin takes all of the work to change the site code to point to your Cloud Files container. Recently a customer had an issue using Magento and didn’t have a tried and true way of CDN-enabling the static content with their site. Magento has a community driven plugin model and has many popular third-party plugins, some which have this capability. The most popular is called OnePica and is available from Magento Connect here: http://www.magentocommerce.com/magento-connect/6274.html[2].

This plugin is a great solution, however it has a few drawbacks. First, it will only sync some of the key category and product images from your site. Anything beyond these are not synched; for instance other files in the skin, js or media. This might not achieve what you would want, which is to offload all of this content from the server. Second, the plugin does not support HTTPS content with Rackspace Cloud Files, even though it supports both HTTP and HTTPS protocols, including some streaming ones to boot. I believe this is just a feature yet to be updated in their plugin.

Rather than being at mercy with a plugin, Magento offers a more native solution that allows you to change the base URL of your site for these three areas, which is for specific use with a CDN solution. This allows you to upload the js, media and skin folders in their entirety to a CDN provider; edit these values to point to the CDN container; and immediately all content is being pulled from the CDN servers instead of from your server. From here, it is just a task of keeping the files that are in these locations on your server in sync with what is in Cloud Files.

For this problem we found an open source program called fileconveyor located on GitHub here http://fileconveyor.org/[3]. This application is written in pure python and works excellent on syncing files from one place to another, specifically to cloud technologies like Rackspace Cloud Files. We forked the project here at Rackspace and figured out a base configuration that works to sync specific directories in Magento with a Cloud Files container. The beauty of the program is that it uses inotify monitoring to watch for changes on the directories, which means that as soon as a file is created, modified or deleted the change is reflected instantly across to the Cloud Files container. This takes the mundane task of manually keeping the files here in sync, as well as extrapolates the responsibility of doing this from Magento or a plugin and brings it to the OS layer where it can better be managed.

This is also a big opportunity for the Rackspace support teams to more easily support this as a solution. The implications beyond Magento are also amazing, as this program can sync anything located on the local filesystem of your server to Cloud Files with ease. We see future use for any CMS that requires cloud files syncing to achieve the benefits of a CDN with less hassle.

If you are interested in implementing this solution for your Magento eCommerce site or CMS, check out this helpful Knowledge Center article[4].

Endnotes:
  1. Rackspace Cloud Files: http://www.rackspace.com/cloud/files/
  2. http://www.magentocommerce.com/magento-connect/6274.html: http://www.magentocommerce.com/magento-connect/6274.html
  3. http://fileconveyor.org/: http://fileconveyor.org/
  4. this helpful Knowledge Center article: http://www.rackspace.com/knowledge_center/article/syncing-to-cloud-files-with-fileconveyor

Source URL: http://www.rackspace.com/blog/easily-sync-server-files-to-the-cloud/