This is a guest post written and contributed by Travis Reeder, Co-Founder and CTO of Iron.io, a Rackspace Cloud Tools Partner. Iron.io provides hosted messaging and background processing services for developers who want to spend less time managing infrastructure and more time building great applications.
Much of the discussion of the cloud stack these days centers on app servers and storage. Whether it's deploying, monitoring, scaling, making fail-safe or redundant, the base model is one where app servers and storage solutions are the core of the cloud stack. This view is certainly a correct one but it's only a partial picture and here's why.
My co-founder, Chad Arimura, and I are cloud developers at heart. When we started working together, we were developing applications for others. Although the cloud was just emerging, we made the fortunate decision to create applications using 100 percent cloud resources. The use of these virtualized resources — namely app servers, NoSQL databases, and object stores — allowed us to quickly build some heavy-duty applications with very small teams. But project after project, it was clear that the core of each app was more than just a set of app servers and data stores.
Even at the onset of building a cloud app — but especially as we needed to grow and scale it — several patterns become clear, 1) all the parts and components of the application needed to be loosely coupled, 2) a lot of processing needed to be offloaded to the background, 3) messages and events needed to be exchanged in ways that were non-blocking and asynchronous, and 4) getting the right flow for the granular pieces — the data and processes — was as important as dealing with the bigger blocks in the architecture.
Cloud Tools and Infrastructure Services
Call it an epiphany that took a few years, but it put us on a path to create services that fit in alongside app servers and datastores to provide infrastructure capabilities. In our case, it’s IronMQ and IronWorker, two services that manage and orchestrate the messaging, background processing and scheduled jobs that take place within cloud applications.
In the model we see at Iron.io, the modern cloud stack as not just app servers and data storage, but also message queues and task queues (and logging and exception handling and a few other core services if you want to be truly complete). There are a number of open source solutions for both message queues and task queues, each of which provide powerful capabilities, but the problem is developers still have to stand up servers and manage scaling, redundancy, fault-tolerance, and deal with a number of other dev/ops concerns. None of the solutions are built for the cloud or handle cloud-loads in ways that scale out seamlessly. It's easy to stand up one instance, but very hard to run two or more.
Our experience as developers made it clear that infrastructure services for these types of systems create tremendous efficiencies and agility — just as you gain with automatic scaling of app servers, automatic replication of databases and a host of other second-generation cloud capabilities.
Making a hosted message queue available to Rackspace developers means that they get to build message handling into their applications right from the start. With a simple API they can create queues and send and receive messages, and not just from Day One but from Minute One. And they can do it without having to worry about managing servers or dealing with infrastructure or system concerns. Having the message queue in the same cloud as their applications and datastores also means that developers get data locality, high performance and security, among other benefits.
The Evolving Cloud Stack
The tools and services that cloud developers are using - along with the patterns and best practices - are the ones that will make up the IT reference model for the next 5 to 10 years. We see Messaging as a Service and Background Processing as a Service as essential elements within this emerging cloud stack. Combining elastic messaging and background processing with scalable app servers and expandable storage solutions is a natural fit.
With this combination, you not only get speed and agility in developing applications, you also get scaling, durability, and the potential for massive capacity. These benefits can accrue to the leanest startups as well as to mega-organizations with multiple applications, large datasets and the need to compete with companies much more nimble.
A cloud stack is the sum of its parts and with IronMQ now a part of Rackspace Cloud Tools, the collection of parts just got much bigger.