An OpenStack Case Study: Enabling the X.commerce Cloud
X.commerce brings together the technology assets and developer communities of eBay, PayPal and Magento to support eBay Inc.'s mission of enabling commerce
Build a robust, scalable, and agile cloud infrastructure that would be able to manageably grow to meet the demands its merchant and developer communities eventually will require
Public cloud options didn’t provide the level of control, transparency, and security eBay needed to build its X.commerce platform
Rackspace Cloud: Private Edition, powered by OpenStack
- Faster feature development and deployment for X.commerce
- Code changes remain compatible with implementation
- New X.commerce features and enhancements can be added with minimal service interruption
- Future OpenStack release support
The X.commerce Objective: Redefine the Marketplace
Technology has forever changed the relationship between merchants and their customers. Buyers today demand choices in how and where they're able to shop, whether that's online, offline, or mobile. Modern shoppers also are armed with more knowledge today than ever before from reviews, friends' recommendations on social media, and instant price comparisons made available through smartphones. These sweeping changes leave merchants of all sizes wondering how they can remain agile and competitive and they're looking for ways new technologies will support them in those efforts.
eBay Inc. believes it is providing the answer through X.commerce, eBay's open commerce ecosystem. The X.commerce ecosystem offers a complete collection of third party commerce capabilities as well as a wide range of technology assets from the eBay portfolio such as technologies from the eBay Marketplaces division, payments provider PayPal, open source eCommerce platform Magento, and the online marketing and commerce services provided by GSI Commerce. Through the ecosystem of developers, partners and tools, merchants now have the ability to quickly implement new technologies for a complete end-to-end commerce experience that allows them to rapidly scale and grow their businesses.
The Technical Challenge
X.commerce is providing merchants and developers on-demand access to that powerful stack of e-commerce technologies. These include dynamic online commerce sites, shopping carts, payment services, and inventory management capabilities. Developers will build their solutions for merchants on X.commerce's platform. And since eBay intends X.commerce to be an open platform, building the platform on open-source technologies is crucial.
To succeed, eBay needed to build a robust, scalable, and agile cloud infrastructure that would be able to manageably grow to meet the demands its merchant community eventually will require.
Execution: Building the X.commerce Cloud Infrastructure
eBay set out to build a cloud platform that, just like its X.commerce set of services, would incrementally and continuously improve over time. While a long, arduous, and linear development process has been the industry norm in previous years, that's just not so in the era of cloud computing. And while many users are familiar with their cloud services and applications being automatically updated and improved without forcing a reboot of their computer or the reinstallation of their applications, few outside of cloud development and operations teams give much consideration to what must go into designing the cloud infrastructure that can be persistently managed and expanded.
Not having to consider the working of their underlying cloud infrastructure was a luxury eBay's X.commerce team could not afford. "Our entire environment must be on-demand. When a merchant or developer checks in code, the idea is to quickly move those applications into production," says Venkata Yella, cloud engineering lead at X.commerce. "That means the underlying infrastructure must be able to expand as needed," he says.
Building the X.commerce Cloud Infrastructure
To build the ecosystem, X.commerce would need to find a cloud infrastructure that could grow to the large-scale, high availability system that would successfully support the thousands of merchants and millions of transactions that would traverse its systems. Also, because X.commerce is built to be open, the cloud infrastructure would need to be open source, community-built, and controlled to avoid any long-term vendor lock-in. Additionally, because of the business criticality of the initiative, and the need for optimal performance and security, eBay decided it would be better served with a cloud architecture it controlled rather than turning to a public cloud provider for the infrastructure layer X.commerce would need.
For that cloud, X.commerce chose Rackspace Cloud: Private Edition powered by OpenStack. Rackspace Cloud: Private Edition is a new approach for customers looking to build medium-to-large-scale private clouds. By leveraging OpenStack, the open source cloud operating system, Rackspace customers can avoid costly software licensing fees and benefit from a continual stream of new and enhanced features being developed by hundreds of developers that are contributing to OpenStack.
The OpenStack cloud provides full control and high scalability in private, dedicated environments that can be almost anywhere-in customer-premise data centers, Rackspace data centers, or partner data centers globally. The Rackspace Cloud: Private Edition is based on a reference architecture and best practices that have been tested for optimal performance at scale, and is supported by the OpenStack specialists at Rackspace. "We chose OpenStack as the infrastructure to power our commerce ecosystem because it is flexible, scalable and best meets the needs of our broad application developer community," says Neal Sample, CTO X.commerce. "Working with Rackspace to build X.commerce on OpenStack was an easy decision, because our philosophies are well-aligned, they are core contributors to the OpenStack project and have a depth of knowledge in hosting and cloud computing."
The X.commerce cloud ecosystem was deployed, by Rackspace Cloud Builders, within a Rackspace data center, initially with 35 servers in two clusters, as an internal trial cloud to test the X.commerce cloud prototype. As part of the initiative, Rackspace Cloud Builders installed and configured the OpenStack Compute, Storage, and Image Registries, completed functional testing. And, throughout the relationship, Rackspace Managed Services for OpenStack will provide full support of the OpenStack cloud orchestration layer, all OpenStack updates and version upgrades, performance tuning, provide security patching and hot fixes as well as escalation support for cloud operations and engineering questions.
The X.commerce stack consists of the hardware layer, the components of Rackspace Cloud: Private Edition, with the web Linux server operating system, Apache HTTP server, and MySQL database all running underneath the open commerce platform. The standard server in the cloud is a Dell PowerEdge r710 quad-core rack server, with 96GB RAM, five 300 GB hard-drives, with about 40 different instance sizes that range from 512MB of RAM with 0GB of additional disk space beyond what is provided by the image, up to 32GB of RAM with 40G of additional disk space.
Following the successful vetting of those test clusters, X.commerce was ready to deploy a set of production clusters for the initial beta production and beta developer sandbox environments. "Now that we have beta production of our sandbox environment, developers can begin to build and deploy their applications to their staging environment," Yella says.
This cloud, as one might assume, architected, built, and maintained by Rackspace Cloud Builders is designed for maximum availability and security. For instance, to optimize security, zones of virtual machines (servers) are established and logically separated by the OpenStack Application Programming Interface (API) and virtual-LANs. These zones are also further segmented at the physical network layer with virtual-LANs. This provides physical network separation between the different layers of the X.commerce ecosystem. This way, the communication among specific zones occurs only if explicitly permitted.
While Rackspace hosts and maintains the cloud infrastructure, the X.commerce team is able to view and manage their environment from their OpenStack dashboard including their physical resource usage, user and tenant management, image and instance management, and security group management. For instance, X.commerce can scale their cloud rapidly, with the ability to automatically add additional nodes without causing downtime.
"One of the aspects we appreciate about working with Rackspace Cloud Builders is that they can keep up with our development team. Seriously. We are a fast-paced, rapidly-evolving organization, and the Cloud Builders can swiftly adjust when we adjust," says Yella.
Those applications and services provided by developers and merchants will be key to the long-term success of X.commerce, helping its retailers to reach more customers, provide buying experiences that build loyalty, and make available the real-time customer data and insights necessary for continued success.
And all of those services and applications will be designed to the X.commerce open ecosystem, running on top of the Rackspace Cloud: Private Edition, to give merchants and businesses one place to access all the benefits of X.commerce, including online storefronts and multiple marketplaces, and expand their marketing channels.
Over upcoming months, X.commerce plans to focus on the scalability and elasticity of its cloud computing environment. "The idea is to build the platform so that merchants can leverage technology to provide standout experiences across all types of devices and in the channel their customers want. That will be achieved by helping merchants manage the entire commerce process, easily and cost-effectively," says Yella.
What Does X.commerce Mean to Merchants, Developers, and Partners?
X.commerce powers the future of technology-powered buying and selling.
X.commerce is a single commerce platform that makes the most current commerce technologies easy to access and implement. Developers can make the technology easy to customize to any merchant's business. And X.commerce's ability to scale quickly without investing in costly, complex infrastructure and applications opens the playing field to merchants of any size, anywhere, to compete and win.
X.commerce provides a robust, open ecosystem that offers integrators and innovators the most complete set of commerce-related tools and capabilities available. Because this ecosystem provides access to third party tools and eBay Inc.'s resources through one entry point, it gives developers access to hundreds of millions of consumers and merchants who are looking for an improved commerce experience.
For Solution Providers and Partners
X.commerce is a new and innovative distribution channel. The open environment makes the newest, and even most competitive, commerce technologies outside of eBay available to plug in - delivering unimaginable choices for developers, merchants, and consumers everywhere.
OpenStack Compute is designed to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform. OpenStack Compute provides the software, control panels, and APIs required to orchestrate a cloud, including running instances, managing networks, and controlling access through users and projects. OpenStack Compute strives to be both hardware and hypervisor agnostic.
OpenStack Object Storage
OpenStack Object Storage creates redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. It is not a file system or real-time data storage system, but rather a long-term storage system for a more permanent type of static data that can be retrieved, leveraged, and then updated if necessary. Because OpenStack uses software logic to ensure data replication and distribution across different devices, commodity hard drives and servers can be used in lieu of more expensive equipment.
OpenStack Image Service
OpenStack Image Service provides discovery, registration, and delivery services for virtual disk images. The Image Service API server provides a standard REST interface for querying information about virtual disk images stored in a variety of back-end stores, including OpenStack Object Storage. Clients can register new virtual disk images with the Image Service, query for information on publicly available disk images, and use the Image Service's client library for streaming virtual disk images.
Web Server Components
Linux operating system, Apache HTTP Server, MySQL database.
© 2015 Rackspace US, Inc.
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License
See license specifics and DISCLAIMER