TrueCar is an online digital car buying marketplace that aims to make car buying simple, fair and fun. The company provides automotive pricing and other data for thousands of buyers of new and used cars in the United States.
The obstacles they faced
Scaling TrueCar's Mission-Critical Price Transparency Engine
Price transparency is a critical part of the TrueCar business model, and the company relies on its Vehicle Intelligence System (VIS) to enable that transparency. The system is a central hub that stores raw car pricing data collected from thousands of dealers, transforms and processes the data, and distributes it to dozens of consumer-facing applications.
As TrueCar grew, managing the VIS became a challenge. “We were running the VIS and other key applications in internal data centers, and the hardware wasn’t keeping pace with the business,” said Tommy McClung, CTO. “We have a small operations team managing thousands of virtual servers. That was getting more difficult to do as we added more dealers to the network.”
The company needed to increase the speed of innovation. “We had 100 software engineers sharing the same QA and development environments for testing, and that slowed the pace of development,” said David Griffin, VP of Technology. “Our product velocity had screeched to a halt. As a company, we needed to innovate faster.”
"Our product velocity had screeched to a halt. As a company, we needed to innovate faster.”
How we helped
Partnering with Onica by Rackspace Technology to Embrace a Cloud-Native Future
For TrueCar, moving to the cloud was the only possible solution to its business challenges. “We continued to see more companies successfully migrating key applications to the cloud, and we knew it was the right time for us to do it,” McClung said.
After evaluating several cloud providers, TrueCar made the choice to go all-in on Amazon Web Services (AWS). “AWS is the category leader, and it had all the services we need to run our application and our company. The choice was very clear for us,” said McClung.
TrueCar worked with Onica by Rackspace Technology, a Premier AWS Partner Network (APN) Consulting Partner that holds multiple AWS Competencies, including the AWS Migration, Big Data, and DevOps Competencies.
“We needed outside experts to help us build the cloud platform we wanted to build, and Onica by Rackspace Technology was the right choice,” said McClung. Onica by Rackspace Technology worked with TrueCar to create a sophisticated big data system powered by the Hadoop open-source framework and AWS.
The VIS system’s inventory data — including car make, model, year and options information — flows through the Hadoop environment, connected to AWS through AWS Direct Connect. The VIS system, several car-buying applications and partner platforms run on Amazon Elastic Compute Cloud (Amazon EC2) instances, with automobile images stored in Amazon Simple Storage Service (Amazon S3) buckets.
Using AWS to power its big data system, TrueCar processes more than 65 billion pieces of data daily, from more than 12,000 data sources, including unstructured data that helps the company detect car-buying trends that show up in clickstreams. The company can also price more than 200 billion separate vehicle configurations every day, with the primary database updated every 30 minutes.
Additionally, TrueCar runs its data pipeline through AWS Lambda and Amazon Kinesis Streams, notifying various car-buying applications when vehicle data is updated and ready for analysis. TrueCar uses Amazon Kinesis Streams to collect and process log data, and Amazon Kinesis Firehose to load the log messages to S3 for permanent storage.
TrueCar runs all of its business analytics and reporting using an Amazon Redshift data warehouse. The company also created internal development and test environments called “space pods.”
These containers for self-service development are managed as Docker containers powered by Amazon EC2 Container Service (Amazon ECS), a scalable container-management service. TrueCar software engineers can use the pods to quickly spin up full AWS environments for experimenting with new features.
“Each pod contains a full instance of the TrueCar application, and it’s a replica of what’s in production,” said Griffin. TrueCar also uses the Amazon Elasticsearch Service to run search capabilities for its big data system.
Using AWS, TrueCar processes more than 65 billion pieces of data daily, from more than 12,000 data sources.
What we achieved together
A New Approach and a Scalable Future
By moving more than 1,000 virtual machines from its data center to AWS, TrueCar can now scale to meet demand without managing hardware. “Even though our operations team is small, we can still keep pace with our fast growth by relying on AWS to manage the environment that powers our big data and vehicle-information systems,” said McClung. “And by shutting down our data centers, we anticipate we will lower operational costs by 50% in the long run.”
TrueCar can bring new features and products to market faster, because its development and test environments give engineers the flexibility to quickly test ideas. “We are no longer limited by turnaround time for buying and setting up hardware,” said McClung. “It used to take us up to six months to set up a new environment for testing a feature, and if the feature didn’t work, we wasted all that time. Now, using our AWS-powered space pods, we can spin up a development environment in 15 minutes to prototype new ideas. This means we can innovate much faster, which gives us a competitive advantage.”
The company’s flexible new development and test environments have also transformed the way operations staff and application developers work. “Previously, things were very siloed. Our infrastructure team focused on the backend, and our application team focused on software,” said Giffin. “Using AWS, however, we now have an infrastructure-as-code approach that has made us all developers. Everyone can collaborate much more easily.”
TrueCar has also significantly improved its data analytics using AWS. “Using Amazon Kinesis Streams and Amazon Redshift, we get near-real-time access to important clickstream data and sales and inventory information,” said McClung. “As we continue to expand our data-analytics platform on AWS, we believe we’ll find new insights from our data to help us drive innovation in the automotive industry.”
About Rackspace Technology
Rackspace Technology is a leading end-to-end multi-cloud technology services company. We can design, build and operate our customers’ cloud environments across all major technology platforms, irrespective of technology stack or deployment model. We partner with our customers at every stage of their cloud journey, enabling them to modernize applications, build new products and adopt innovative technologies.
Let’s Talk Strategy
Tell us a little about your challenges and we’ll contact you.