By EJ Johnson
Today marks the release of a new feature to Cloud Files as well as a couple of other changes we wanted to highlight.
CDN LOG RETENTION
This new feature gives customers the ability to save their raw CDN web logs to their Cloud Files storage account.
One of the biggest attractions of Cloud Files for our customers is the ability to serve files over Limelight Networks. By partnering with Limelight, a tier one CDN provider, we’ve put real CDN power into the hands of everyone so they don’t have to worry about signing contracts or setting up additional infrastructure. One drawback we’ve had was having visibility into their CDN usage. Now, by offering the ability to store your raw CDN web logs, we are taking our first step at providing more insight into your usage of Cloud Files and CDN.
A couple of details about this new feature:
• This is an “opt-in” feature. Log files will only be saved if this new log-retention setting has been set on a CDN-enabled Container. We recognize that most users will want access to these logs; however, since the data will be stored in their Cloud Files storage account, we didn’t want everyone to start paying for additional storage unless they choose to.
• There is no automatic purge of saved logs. Once logs are saved to Cloud Files, they will stay there unless explicitly removed. As we’ll discuss below, the Object name of each log file includes a date/timestamp so it would be fairly straightforward to set up a weekly or monthly script that purges old logs.
• Although logs will be periodically saved to the storage account, there is no predictability as to when logs will be processed and saved. It depends greatly on the accumulation and transfer time from Limelight and also on the processing time of Rackspace systems. The time between a “hit” of a CDN-enabled Container/Object and the time for that web log entry to appear in the storage Container could be anywhere from 8 to 24 hours.
• Once enabled, logs will be saved to a Container named “.CDN_ACCESS_LOGS“in the customer’s storage account. The Object names for each log file will have a unique name like: container_name.YYYYMMDDHH-XXXX.log.gz. The “container_name” will be the name of the CDN-enabled Container. The next part of the name represents the year, month, day, and hour that the log file was processed. The XXXX part is a zero-padded 4 digit number that prevents name collisions if multiple logs filtering in from various Limelight geographic locations during the same hour period.
• The log files are stored in [gzip - http://www.gzip.org] compressed format to save on storage and bandwidth costs. The naming convention allows the logs to be sorted lexicographically by Container name and then by date.
• This feature can be enabled via the ReST API of the CDN management system or via any of our supported language API’s hosted on github.com.
• We expect third party tools and other language bindings to add support for this feature shortly. Please see the Developer Guide for the specifics regarding the ReST API or the documentation included in each language binding.
NEW CDN URI FORMAT
There has also been a change to the URL format of a CDN-enabled Container. Previously, when a Container was CDN-enabled, the URL looked like this: “http://cdn.cloudfiles.mosso.com/c10171″. The new format is: “http://c00010171.cdn.cloudfiles.rackspacecloud.com”. Of course, one change is the hostname part of the URL to align with our re-branding. The other change shifts the “unique” identifier from a path element to a prefix of the hostname. The benefit to customers is mostly edge-case. It provides the ability for customers to serve CDN content at the root-level of the URL rather than dangling off a path element. This is critical for Flash players needing to specify a cross-domain policy that enables serving Flash content from their site via the Cloud Files CDN.
Both URL formats will work. We realize that many of our existing customers have probably directly referenced the old CDN URL format in web sites, blog posts, etc. Just be aware that the new format will be returned via the API even on Containers that were CDN-enabled prior to this change.
MULTI-FILE UPLOADER IN CONTROL PANEL
There has also been a recent update to the Cloud Files web interface in the Control Panel that allows users to upload more than one file at a time. Users also have the ability to adjust the TTL setting on a CDN-enabled Container. Valid values for the TTL setting are between 1 and 3 days.
There have been a few bug fixes included in this release, most notably to resolve issues with persisting/viewing Object metadata.
We will be issuing a follow-up release very shortly that will also allow users to enable CDN access log retention from the control panel.
Currently this new feature is only configurable with the ReST and language APIs.
WHAT’S NEXT FOR CLOUD FILES?
We are always listening to what our customers’ needs are. Having access to CDN logs was one of the most highly requested features for Cloud Files. There are still a few remaining “most requested” features that we are working on. The new CDN URI format change is a big step as it sets us up to provide true CNAME support for CDN-enabled Containers. Rest assured, we’re still working to make that available as soon as we can.
We’ll also be coming out with another feature with respect to the CDN component of Cloud Files. I don’t want to reveal the details yet, but we expect this to be a very important feature to Cloud Files customers using the CDN. Stay tuned.
Have more questions?
Feel free to hit us up on chat or give us a call if you have other questions. You are also welcome to join us on IRC for “unofficial” support and talk tech at irc.freenode.net on #cloudfiles.