Let’s look at some of the considerations and long-term risks of adopting a platform:

• Obsolescence – It is surprising to see how quickly much technology trends change. Some technologies that used to lead a market are now extinct, while others are used because they are the only option. Remember dBase III, Cobol and WordPerfect? When a technology is supported by a vibrant community it has a much lower probability to become obsolete than when the strategy is driven by a single corporation and follows a single vendor business strategy.

• Availability of skills – There is no such thing as an overskilled or overstaffed IT organization. CIOs are challenged to find quality talent in areas like network security, virtualization, DR and other areas. A technology that is more familiar to a broader staff is easier to adopt and to use properly. Open technologies allow CIOs to reuse their skills across different kinds of development and ensure the same set of skills can still be leveraged as the organization uses different implementations of an open technology.

• Vendor choices – Most CIOs cringe when it is time to renew multi-million dollar software licensing contracts. But the real pain is when that software is no longer sufficient or does not meet the organization’s business needs, yet it is so ingrained that the option of re-writing to another, better platform seems desirable despite being incredibly expensive and laborious. Open platforms provide an option to build to a standard that allows vendor flexibility. Looking at history, while the write-once run-anywhere promise of Java never materialized in full, a large number of enterprise applications were built on the J2EE platform knowing that moving from one Java container to another one was not only possible but practical despite implementation differences. Similarly, in the cloud, an application that has been designed for the OpenStack API is likely to be able to run unchanged, or with minimal changes in multiple OpenStack-powered clouds. This provides freedom to switch vendors if or when a particular vendor is no longer meeting the business needs or has unfavorable pricing or support services.

• Architectural flexibility – One of the challenges of a single vendor strategy is the need to align with the architectural choices of a single vendor. There are many ways to implement technology and vendors, so it is only feasible to support limited choices. An open cloud gives IT the opportunity to choose between multiple implementations of a technology: spinning or SSD drives, automation versus customization, burstability versus predictability as examples. In addition, an open cloud allows an organization to adopt a multi-cloud strategy.

• Business model flexibility – The cloud is an enabler of innovation, not only in the way it is consumed but also in the way it is offered by vendors. An open cloud empowers IT to choose the right pricing, support, services and deployment model. IT can select on-premise, private hosted, public or hybrid; standalone or integrated with dedicated or virtualized infrastructure; and a model optimized for CapEx or OpEx. An open cloud provides more implementation choices. The public cloud from one vendor is unlikely to be the right answer for all customer needs – no cloud is one-size fits all.

• Technology flexibility – Similar to the architectural flexibility concept, vendors use their own point of view and profit interests to guide and evolve their technology and make decisions that will impact the future of a platform. For a technology as broad, complex and fast-changing as cloud, it is hard to imagine a single vendor has the knowledge, expertise and vision to make all the right decisions for the future of a platform, even if they have customers’ interests as their top priority.

In the open cloud, innovations are provided by multiple contributors. Companies like Cisco, EMC, HP and IBM contribute their knowledge in their respective areas of expertise like networking, storage, high performance computing and others. Customers who make large investments in private clouds are also contributing their innovations to OpenStack. The result is a platform designed by the brightest minds in the industry that provides multiple options and implementation choices that are more likely to meet IT requirements. OpenStack decisions are made by a board that comprises vendors, industry experts and customers to ensure the platform evolves in a way that benefits the community as a whole and not a particular vendor.

For more information on the open cloud and technology adoption considerations, check out the new white paper “Welcome to the Open Cloud.”

I was compelled to answer. I started with the idea that the main challenge IT departments face in terms of innovation is that IT is hard and it’s not getting any easier. IT is increasing in scope in almost every organization and it is becoming more complex every day. IT departments are under-resourced because they are thought of as a necessary expense to run the business, but not an engine for innovation.

The increased business needs for IT infrastructure with limited resources results in a backlog of tasks that IT has to perform; but it can’t. IT is prevented from helping the business move at the speed that it needs to. An example is a marketing department that needs a new content management platform to launch a new product, or a sales department that needs a better collaboration and learning platform to increase sales effectiveness. IT is struggling to keep up with these business needs; which are imperative to the business’ overall success.

Some stats show most IT departments spend as much as 80 percent of their time and resources just keeping the lights on. IT in a medium-sized company might manage hundreds of applications and each one needs monitoring, patching, updating, maintenance, etc.

The cloud can help inject innovation into IT in two major ways:

1. Infrastructure Efficiency

By making infrastructure virtual and based on pools of resources, the cloud adds agility and improves resource utilization.

Historically, if a business unit needed for an application that required a few servers, there was a great deal of red tape to wade through: it had to work with finance to approve the spend; then with procurement; then it had to wait for the equipment to arrive; find space in the datacenter; ensure there is cabling, networking and power; set up the servers; then set up the application. (Phew! Are we done yet? Not even close!)

Once all that was taken care of, one of two things happened: Either the servers were not powerful enough to handle the number of users for the application, resulting in a bad user (or customer) experience; or the number of users was lower than the number of users supported by this server, resulting in unused capacity.

With the cloud the whole process of acquiring and setting up a server can be reduced to minutes. Even installing and configuring applications can be done swiftly. For example, a business that needs a collaboration portal can go to the Rackspace Cloud, create a Cloud Server with a SharePoint image and have the site up and running in an hour.

Because the cloud is elastic, the server can be resized as needed (larger or smaller depending on the use). Once the application is no longer needed, it can be turned off, and the meter stops running.

For more complex applications, Cloud Monitoring can alert an administrator when a server is not responding fast enough. With more advanced monitoring, admins can be notified when a server is reaching CPU or memory usage thresholds. The user can make the server larger, or can create additional servers that could join a web server cluster via a load balancer, for example. All this can be done within minutes from a mobile device, a tablet or other web-enabled device.

Ultimately, by using more agile and efficient infrastructure, IT can turn its attention away from simply keeping the lights on and focus on spurring innovation within the business.

2.  Skills Effectiveness

Every IT department is working with fewer resources than it needs. Finding skilled talent is incredibly difficult, probably harder than buying servers. Maintaining a complex web application may require a database administrator, a security expert, a network administrator and a systems administrator. Hiring a team like this can take months and can require a massive chunk of payroll.

And that is if you can find the talent. According to an IBM report, in Canada alone the IT industry will be more than 100,000 workers short by 2016 thanks to growing skills shortages that affect as many as 60 percent of all businesses.

One solution to that problem is the outsourcing of some of the infrastructure management. While many cloud infrastructure providers offer just basic infrastructure, others like Rackspace provide a wide breadth of managed services.

For example, at Rackspace we have teams of specialists that can help design, plan, implement, manage and maintain infrastructure for our customers. Taking advantage of managed services means there is a team of people monitoring your infrastructure, installing updates, proactively tuning it, troubleshooting and taking care of other administrative tasks, like this recent blog post illustrates. There is someone making sure your application works 24x7x365.

For customers running business-critical applications, Rackspace also offers Critical Application Services that include a 100 percent uptime SLA guarantee for the application platform or for specific applications such as Adobe CQ for content management.

Outsourcing repetitive tasks and infrastructure management allows an IT department to spend less time on maintenance, hardware infrastructure and management tasks, freeing up more time to innovate.

The model has proven across large IT environments, SaaS companies and online retailers. Chris Le, Lead Developer at Seer Interactive explained it in very practical terms: “Managed Cloud really allows us to focus specifically on our app, and not all the IT stuff around it. Not that we can’t do it, but I have code to write.”

The value of taking advantage of both infrastructure and skills efficiency in the cloud is probably more evident in startups, some of which have no IT department at all.

In a post titled “Servers are cheap, talent is expensive,” Lydia Leong, Gartner’s leading cloud analyst, explains: “If you can outsource a capability that doesn’t generate competitive advantage for you, then you can free your best people to work on the things that do generate competitive advantage.”

Analysts are bold in their predictions. Gartner, for example, predicts that “Big Data will deliver transformational benefits to enterprises within two to five years, and by 2015 will enable enterprises adopting this technology to outperform competitors by 20% in every available financial metric.”

In the same report, Gartner places Big Data near the “Peak of Inflated Expectations” in the hype cycle, which can be defined as a phase that generates high amounts of enthusiasm and unrealistic expectations (i.e. what most people would call a buzzword). Given the current hype, it is useful to take a step back and understand where these technologies can be useful and try to distinguish hype from reality.

One aspect of the vision for Big Data is related to business intelligence applications, which seek to empower businesses and organizations to derive intelligence and insights that will enable them to act smarter, resulting in a significant competitive advantage. New forms of processing are needed to deal with the three core characteristics of Big Data (from Gartner’s own definition): high volume, high velocity and / or high variety of data.

Solutions such as Hadoop and NoSQL technologies facilitate storage and analysis of very large, unstructured data sets that have been challenging to manage with traditional SQL databases. While these technologies solve a significant part of the burden associated with business intelligence efforts, there are two key problems that still need to be addressed.

The first problem is that it is still highly complex to source and integrate enterprise data. Extracting, de-duplicating and correlating data about, say, customers and profitability, continue to be monumental tasks, particularly because they tend to involve a large number of source databases and information systems with potentially different definitions for the same piece of information.

The second problem is probably the harder one to solve because it goes beyond technology and into the skills available to the organization. Having access to an incredible amount of data and the ability to do complex queries are only part of the problem. To produce business value, one must derive insights from the data and be able to act on it. For example, marketers seldom act on the data available to them. In my experience, most marketers (with the possible exception of those companies that extract direct revenue from website visitors via online retail or advertising) rarely look at web analytics data and therefore fail to act on any insights that these tools may offer.

Regardless of the challenges, Big Data can be incredibly powerful when properly applied, but it will require expertise and skills that may not exist today in many enterprises (which is expected with any new technology). In addition, the tools used to visualize, query and summarize data will need to mature. Given the interest in this technology, I expect both of the challenges discussed above to be solved quickly by the industry.

What seems to be lacking is a deep understanding of the type of problems Big Data is designed to solve. Big Data or NoSQL technologies will not replace traditional databases that are designed to maintain relationships between structured data sets and to perform operations such as transaction processing that require the ACIDity provided by SQL (Atomicity, Consistency, Isolation and Durability of transactions).  SQL databases will continue to be fundamental technology tools for many, many years.

From a market perspective, Microsoft SQL Server’s revenue is roughly $2.5 billion and grew by 20 percent in the last year. Meanwhile, the total revenue for NoSQL databases, which, according to The451, reached just $20 million in 2011. The451 expects the total NoSQL market to grow to $215 million by 2015, which is still less than half the growth in license revenue that SQL Server saw in 2011. I use this comparison only to highlight the sheer volume of problems that are still the sweet spot for enterprise mission critical applications backed by relational databases.

The main point is to set the right expectations. As it is usually the case, it is about selecting the right tool for the job, as GigaOM points out in the article “MongoDB or MySQL why not both?” Because NoSQL databases give organizations the advantages of scale and flexibility of data structures, they are a good tool for managing large amounts of data where the relationship between the data elements is less important.

As the Wikipedia article states: “NoSQL database management systems are useful when working with a huge quantity of data and the data’s nature does not require a relational model for the data structure.” To choose the right tool for your data problem, you should try to understand the business requirements across three dimensions of size, variety of the type of data (unstructured versus highly structured data) and velocity of ingestion and removal of data. In addition, NoSQL databases can often be deployed using commodity hardware, making it an affordable technology to deploy from a hardware requirements perspective.

I propose that there are three key aspects of NoSQL and Big Data technologies that we should remember:

1. Organizations should choose database technologies based on the business requirements and the problem at hand. This requires understanding the virtues and challenges of each technology, and to fight our natural inclination to favor “cool technologies.”
2. No single information management technology is the right solution for all needs, whether it is SQL, NoSQL or any other. NoSQL and Big Data offer organizations very high value for specific business and technology problems that require high amounts of varied data types with high velocity of change. Some examples include log analysis, transaction analysis, very large data sets and many applications that require computations and analysis that are impractical to perform in a relational database.
3. Most organizations will struggle to realize the utopian vision that Big Data will deliver unlimited customer insights and “automatic” business value. It is not a panacea. It is still important to ask the right questions and be able to act on insights, and to develop the right skills across the organization.

Rackspace has been involved in NoSQL technololgies for quite some time (interestingly, a fellow Racker coined the term NoSQL). Our own IT department has deployed a NoSQL cluster on its own OpenStack private cloud to provide the business intelligence our management team needs (stay tuned for details).

At Rackspace we operate under a fundamental principle of openness, which means that we should support the technology choices of our customers. Whether you need MySQL as a service, SQL Server on dedicated or cloud infrastructure or a NoSQL cluster using technologies from our partners (such as Mongo or Infochimps), we aspire to offer you the right tool for your job.

1. File Storage and Sharing

Chances are, you have suffered the frustration of trying to send or receive a very large file via email. Services like SendFile, Dropbox and Rackspace’s own JungleDisk have been around for years. Apple is helping the industry by explaining the concept of the cloud to the general population.

The year of 2012 brings you mature and powerful options for secure file sharing for workgroups, team collaboration and file storage and backups. From Cloud Drive to hosted SharePoint, there is no reason to send large attachments anymore. Your virtual team, including partners and customers, can have a place in the cloud where they share information and documents.

2. Cloud Database

Almost every web application needs a database. In the past, web developers had to set up and maintain their own databases: MySQL, SQL Server, etc.  Managing and tuning a database requires a very specific skill set to perform complex tasks. This work is best done by a database administrator.

Cloud databases give developers and IT a powerful and scalable database that just works. From the infrastructure to the database software to the tuning and monitoring – everything is done as a service. Rackspace offers multiple options: from a web application platform that includes database functions to a fully managed database server to cloud based solutions from partners such as Xeround.

3. CRM

Companies are adopting the cloud for mission critical applications. Customer Relationship Management (CRM) systems are mission critical and deal with two of the most sensitive pieces of data: customer information and revenue.  Yet, most companies don’t even consider hosting their CRM applications on premise. We have seen Salesforce.com and others grow exponentially over the last few years. The advantages of cloud-based CRM systems are clear for Sales teams and IT and the risks have been addressed. CRM can become a blueprint for moving more line of business applications to the cloud.

4. Email

Email is another mission critical application that is fairly mature and standardized, prime to move to the cloud. For years, CIOs have expressed a desire to outsource email. In 2012 it makes no sense to host an Exchange server under a desk at a company of any size. Independent professionals to large multinational companies should move their email to the cloud as soon as possible.

At a personal level, we all use cloud-based consumer email services like Hotmail and Gmail, and we enjoy the benefits of access anywhere and not having to think about capacity or server uptime. There is a right cloud email solution for every business, be it hosted low-cost email, Hosted Exchange for small businesses, or even managed dedicated Exchange environments for large customers. Hosting your own email is so 2000s.

5. PaaS for Web Applications

Platform as a Service (PaaS) became a buzzword in 2011. Like many buzzwords, it is probably overhyped, yet there is real value in the concept of a PaaS. A PaaS solution allows developers to host their applications without having to think about servers at all: they just upload their application and it runs.

The caveat is that the application ‘stack’ is a black box that has predefined components and settings. This makes a PaaS, like Cloud Sites, ideal for web developers, microsites and stand alone applications that don’t need stack customizations and don’t interact with other line of business applications.

6. File Backup

Everyone should be backing up all important documents and files, but few of us actually do it consistently and efficiently. A good backup stores a copy of your files at a remote location. Until recently, that meant making a backup to a disk or tape and shipping those to a storage facility which was logistically complex, time consuming and not very cost effective.

Cloud-based backup is a powerful solution: backups can be scheduled to run automatically, information is stored in a secure remote location where it will always be available when needed and capacity is never a problem.

7. Web Site Hosting

Websites can be a huge drain on IT resources, especially if the websites are visited frequently. Hosting the website in the cloud combined with managed services allow web teams to focus their efforts on creating the best web content possible (instead of sweating the ins and outs of web hosting). From a simple blog site to a high trafficked corporate website, cloud-based web hosting can provide scalability and high availability when designed properly, which leads right into…

8. eCommerce

Scalability and availability are critical concerns for online stores. Every minute of downtime can result in lost sales. A slow website can result losing a customer for life. eCommerce is also known for seasonal high peaks, such as the holiday season. Online stores no longer have to pay for and deploy infrastructure to support peak times, the cloud allows them to dynamically scale as their traffic scales.

When a spike in interest at an eCommerce website occurs, the demand could bring down or significantly slow down traditional servers, preventing customers from being able to make a purchase. However, the cloud allows that same website to quickly spin up additional resources and handle the load. When the rush subsides, those resources are turned off.

The cloud also makes it easy to deploy web servers in different locations with load balancing to accelerate local page load times and increase availability. Adding the services of a Content Delivery Network (CDN) makes it easy to distribute high-bandwidth content like images and video across the world in a very efficient and cost effective way.

9. Test & Development

A software company that is deploying a new line of business software, or building a software application to sell in the market, usually requires two or more ‘QA environments’ – each one being a setup of servers, storage and networking. Instead of buying and maintaining these QA environments, software development teams can create them in the cloud.

Development teams will benefit from the agility of creating instances in minutes, the efficiency of paying only for the infrastructure needed at any given point in time and from the efficiency of not having to manage and maintain the infrastructure. Additional instances can be set for testing and training purposes with the same efficiency. Load testing and simulation under different hardware configurations is also a no-brainer for the cloud.

10. Private and Hybrid Clouds

Private clouds give IT departments many of the benefits of the public cloud with the added benefit of having an isolated network and computing resources that bring additional security. Rackspace offers private clouds that give IT departments a lot more control over the resources deployed and the architecture.

Hybrid clouds, enabled by technologies such as RackConnect™, allow IT departments to connect public cloud, private cloud, dedicated hosted and on-premise infrastructure to gain the optimum combination of control and agility. For example, a web server can be set up in the public cloud, transaction processing can be in a dedicated server where PCI compliance is easier to attain, order processing can be on-premise and the ERP system can be on a private cloud with automatic backups going to storage on the public cloud.

The cloud is not a go/no-go decision for IT departments and small businesses; however, the thinking should be about how to maximize the value of the cloud.  It is hard to find a consumer, small business or IT department that does not use the cloud in some shape or form.