Opscode, Open Source Configuration Management

Filed in by John Willis | June 24, 2010 10:34 am

This post was written and contributed by John Willis, VP of Training & Services for Opscode[1], a Rackspace Cloud Tools Partner.

This is a special blog article for me today.  Nearly three years ago I started hosting my blog[2] on what was then called Mosso (now called Cloud Sites[3]).  At that time Lew Moorman[4]–now President, Cloud and Chief Strategy Officer–was just another cloud enthusiast like me. He generously gave me a free account to test out the Rackspace cloud offering.  In fact my first <put name here> effect happened soon after and I wrote this blog article[5] commenting on how great Rackspace was for elasticity.  Since then I have maintained a great relationship with the Rackspace folk, doing Cloud Café podcasts and prophetalizing their great services.

When our Rackspace friends asked us to write a blog article in regards to our limited beta release of the Opscode Platform[6], I was thrilled.  Being a fan of Rackspace and being extremely excited about our “Opscode Platform“ offering I thought this was a great opportunity to explain why our Opscode Platform announcement with Rackspace is so important.

I have been waiting for this set of events for decades (true ephemeral infrastructure – cloud – and infrastructure automation).  I often joke with people saying: “Opscode is the first real job I have had in 30 years.”  The truth is that I have been an operations junkie for thirty years waiting for the rubber to actually meet the road.  Working for Opscode and solving huge infrastructure at scale problems and working with a great cloud offering like Rackspace is a dream come true (too bad I had to wait 3 decades for this to happen).

Last year I fell in love with Chef[7], the open source project developed by Opscode. I knew then that I needed to lobby Adam Jacob, Opscode CTO and Jesse Robbins, CEO to join their team. For thirty years I have been hearing promises from large vendors with phrases like Infrastructure Automation, Autonomics, Policy Based Orchestration–however I had never seen the correct tools and infrastructures to provide the framework for these ideas.  About six months ago after relentlessly pestering Adam and Jesse they finally gave in and made me the VP of Services and Training.

When people ask me why I am so excited about working for this Opscode including a world class team –  I usually give them two main reasons.  First, I believe that open source frameworks, like Chef, for configuration management and systems integration has been a long unanswered need–taking one-off scripts and socializing them through the enterprise has been needed for a long time, most any sysadmin can tell you that’s a problem.  Second, running configuration management as a hosted service is the right way to manage infrastructure in 2010 and beyond and consistent with the adoption of fully hosted infrastructure in the cloud. I like, Rachel Chalmers’, an industry analyst at The 451 Group, analogy “Itʼs [Opscode] the salesforce.com of configuration management.”

With the permission of my good friend Mark Hinkle at Socializedsoftware.com I am going to use some of his excellent commentary about Opscode and Chef[8] and how we can solve the current challenges of server configuration…

____________________________________________________________________________

The Challenges of Server Configuration

A large part of configuring infrastructure is repetitive and time consuming. Many sysadmins rely on their own scripts to help manage the process but it’s still a one-off for each administrator and their individual knowledge seldom if ever gets institutionalized throughout the entire organization.

Having a framework for maintaining configuration data is important. Configuration scripts are often authored as shell scripts or Perl or even Python but are seldom maintained or used beyond the original author. Some technologies use plugins that could be used to abstract configuration variables; users and systems then shared among users of the technology but plugins are often specific to a given technology and non-transferable.

Despite the breadth of this problem there are relatively few solutions that can easily be consumed by medium-sized enterprises outside of large management suites available from the Big Four (HP, IBM, CA and BMC). The independent software vendors have all been consumed by bigger less focused organizations including Opsware[9] (acquired by HP), Bladelogic[10] (acquired by BMC) and Configuresoft[11] (acquired by VMware). In my opinion there is no real leader in this space.

Opscode CEO, Jesse Robbins[12], notes in his experience maintaining availability as a top ops guy and “master of disaster” at Amazon.com he had no access to these tools, they were simply sold in a way that was inconsistent with the way he evaluated and consumed products and services.  Opscode, a relatively new company develops the open source Chef project, which automates IT management via a client-server platform.

Opscode’s approach to server configuration challenges is to use recipes written in Ruby, the chef domain specific language (DSL). Then these cookbooks can be executed securely by the Chef client-server architecture and finally Chef is available as open source software to download[13], use and redistribute. In a nutshell Opscode met the following criteria is what interested me about their technology, which is relatively easy to use, share configuration recipes and consume.

Chef, The Open Source Project

Chef [14]is a systems integration framework released under the Apache License Version 2.0.  Chef, can manage servers by writing code in Ruby stored in configuration recipes called cookbooks[15]. Chef can integrate with existing infrastructure like LDAP via libraries [16]using arbitrary Ruby code, either to extend Chef’s or to implement custom classes. Users can also configure applications that have dependencies on other parts of the infrastructure like databases and discern that information via the Chef server. However, I like Robbins’ description of Chef — a sysadmin robot performing configuration tasks automatically and much more quickly than a single admin could ever hope to.

Though Chef was only released on January 15t, 2009 it has gotten rapid adoption and gained a large number of contributors. According to the Opscode wiki there are 157 approved contributors[17] to Opscode projects and well over 20 companies. Beyond that the #chef IRC channel is typically attended by over 100 users and Opscode staff, signs of a healthy, growing open source community.

Opscode, The Platform

The Opscode Platform[18] is the commercial offering from Opscode Inc. It is a centrally managed data store hosted by Opscode into which servers publish data such as IP addresses, loaded kernel modules, OS versions and more delivered as infrastructure-as-a-service (IaaS). This data on the Opscode Platform can be accessed and becomes useful in the following ways:

The Opscode Platform is in a free beta release for the next 60 days. After the trial period, participants can manage up to 20 nodes on the Platform for $50 per month and $5 per month for each additional node. Pricing and availability information is available on the Opscode website.[19]

Opscode, The Company
Opscode recently announced that the close of an $11 million Series B round of funding. The round was led by Battery Ventures (who also was an investor in BladLogic before it was acquired by BMC) and includes a follow-on investment from Draper Fisher Jurvetson(DFJ) whose other open source investments include SugarCRM and Fonality. DFJ led Opscode’s Series A round of funding of $2.5 million, bringing the total amount raised for the company to $13.5 million, a sizable amount of capital to bring this technology to market.

Opscode is also seeing good adoption of Chef, not only do they have a few thousand active users on their wiki plus chef is currently in production at numerous top websites, including 37Signals, Etsy, IGN Entertainment, Scribd, and Wikia. Not only are web jockeys using chef but other large infrastructure providers are contributing to the project. Engine Yard, Rackspace, RightScale and the Springsource division of VMware have signed on to contribute to the project. They are even being very public about it as seen in this endorsement:

“We are excited about the open source contributions the Springsource Division of VMware has made to Opscode Chef.” said Javier Soltero, CTO of Springsource Management Products at VMware. “Chef is an important tool for automating infrastructure management and we look forward to its continued growth and success.”

Making Sysadmins into Superheroes

Opscode Chef is a hugely powerful tool that can greatly amplify the knowledge an effectiveness of systems administrators by automating a significant number of their maintenance tasks, improving their productivity and allowing them to focus on higher value tasks. Not only does Chef provide a framework for building systems but repairing them, keeping availability high and time to resolution low. This gives IT professionals a lot of leverage in getting their tasks done, allowing them to solve a problem once and then automate the process going forward. In other words Chef can turn systems administrators into super heroes by vastly improving their productivity and overall quality of service.

The need that Opscode addresses can be filled to some degree through other software. Cfengine[20], Puppet[21] and bcfg2 are all open source software solutions that address server configuration needs and have been around for some time. As mentioned above there is also large management suites that handle the same problems though they are expensive and have their own limitations. What is unique about Opscode approach is that they offer a robust fully featured software platform as open source and a commercial offering that has full compatibility with the open source project.

This is somewhat unique as many commercial open source projects have a specific feature set that are only available to their enterprise customers. In a conversation with Opscode VP of Service, John M. Willis we discussed those users who are not interested in the Opscode platform but still want commercial support. He said that Opscode is building a select high-quality partner network that can handle these requests. Most recently Opscode has announced a partnership with DTO Solutions who employ members of the Control Tier project http://controltier.org/wiki/Main_Page and is a big proponent of the DevOps[22] approach to infrastructure management. Other partners will soon be onboard as well.

I am very much a fan of Opscode and their approach though my description probably doesn’t do it justice. Theoretically, a systems administrator who successfully implements automation tools such as Opscode can improve not only their productivity but the uptime of servers by improving speed to resolution for outages.  Opscode has a great opportunity thanks to a talented team, a novel go-to-market plan and a real need for these types of tools among IT professionals.

Endnotes:
  1. VP of Training & Services for Opscode: http://opscode.com/team/
  2. my blog: http://www.johnmwillis.com/
  3. Cloud Sites: http://www.rackspacecloud.com/cloud_hosting_products/sites
  4. Lew Moorman: http://twitter.com/lewmoorman
  5. blog article: http://www.johnmwillis.com/amazon/does-your-hosted-provider-give-you-this-kind-of-elasticity/
  6. Opscode Platform: https://cookbooks.opscode.com/users/new
  7. Chef: http://www.opscode.com/chef/
  8. commentary about Opscode and Chef: http://socializedsoftware.com/2010/06/21/opscode-turning-sysadmins-into-superheroes/
  9. Opsware: http://en.wikipedia.org/wiki/Opsware
  10. Bladelogic: http://en.wikipedia.org/wiki/BladeLogic
  11. Configuresoft: http://en.wikipedia.org/wiki/Configuresoft,_Inc.
  12. Jesse Robbins: http://radar.oreilly.com/jesse/
  13. download: http://radar.oreilly.com/jesse/
  14. Chef : http://wiki.opscode.com/display/chef/Home
  15. cookbooks: http://wiki.opscode.com/display/chef/Libraries
  16. libraries : http://wiki.opscode.com/display/chef/Libraries
  17. 157 approved contributors: http://wiki.opscode.com/display/opscode/Approved+Contributors
  18. The Opscode Platform: http://www.opscode.com/chef/
  19. Opscode website.: http://www.opscode.com/pricing
  20. Cfengine: http://www.cfengine.com/
  21. Puppet: http://www.puppetlabs.com/puppet/introduction/
  22. DevOps: http://www.jedi.be/blog/2010/02/12/what-is-this-devops-thing-anyway/

Source URL: http://www.rackspace.com/blog/opscode-open-source-configuration-management/