Support: 1-800-961-4454
1-800-961-2888

How BuiltLean Survived The Front Page of Reddit, And The Traffic That Followed

6

It was February 14 and BuiltLean was about to have the biggest traffic day in its two- year existence. And we weren’t ready.

A post on posture problems that develop from office work was submitted to the LifeProTips subreddit at 10 a.m. Eastern. Within the first hour, it had garnered 500 upvotes and 35 comments; by the second, it reached over 1,000 upvotes and had hit the front page.

Previously, we had survived hitting the front page twice on Digg, largely in part because we had hit during off-peak hours. But it was a Thursday at noon, and there was no avoiding a lot of traffic arriving at once from Reddit.

Like the once fabled “Digg Effect,” Reddit has the ability to send a tremendous amount of traffic to your site. What makes this type of traffic especially difficult to handle is the rapidity and frequency in which it arrives – it often resembles a small-scale DDoS attack. It’s like a machine gun. We were prepared to handle 100,000 hits in a day, but we were nowhere near prepared for 100,000 hits in an hour.

At the time, BuiltLean had a pretty simple set-up that was optimized completely on the CMS side: our HTML, JavaScript and CSS files were minimized and compressed; all PHP scripts and pages were cached; and most elements of the site were served through our CDN. However, we had two significant blind-spots, which were a lack of an HTTP accelerator and a server set-up that was woefully underpowered to handle such a traffic surge.

Luckily, through active monitoring of our referral traffic, we spotted the Reddit submission within the first hour. When we knew we weren’t ready for what was potentially about to happen, we called up Rackspace support, which quickly diagnosed the situation and prescribed an upgrade of our database server and the temporary addition of a second server with a load balancer to distribute requests between our two servers. Within 20 minutes, both were implemented, almost simultaneously with our post hitting the front page of Reddit.

Thanks to Rackspace’s swift deployment, we were largely able to absorb the amount of traffic sent from Reddit. All told, we received over 125,000 visitors from Reddit, 60,000 of which came between noon to 2 p.m. But the on-the-fly adjustments to weather this traffic storm weren’t a long-term fix for the underlying issues. We had to be ready in case something similar happened again.

Going forward, in collaboration with Rackspace’s support staff, we made two substantial changes:

  1. We added Varnish, an HTTP accelerator that greatly reduces the time in which pages are served. When we did have errors during the Reddit traffic surge, it was largely due to maxed out simultaneous connections in Apache that were exacerbated by the slow distribution of the page requested.
  2. We permanently added a second server with a load balancer that acted on-demand during our most trafficked hours, and we could activate it when situations demanded it.

We’re happy to report that with the above implemented and constant refinements done to BuiltLean’s WordPress configuration, page speed and sever stability have greatly improved. Akin to the saying “dress for the job you want, not the job you have,” our takeaway was to always be prepared for the traffic we wanted.

About the Author

This is a post written and contributed by Alexander Baldwin.

Alexander is Director of Outreach for BuiltLean, a Rackspace customer. BuiltLean is a fitness resource for busy professionals who want to get lean and stay lean with time-efficient workouts. The company provides free medically-reviewed articles and videos accessed by over 1 million visitors per month alongside an eight-week fitness program used by customers in over 90 countries.


More
  • Christian Ashby

    ‘Like the once fabled “Digg Effect,”’ – in my day, it was called the being slashdotted. Does that make me a dinosaur?

  • http://www.iwordsofwisdom.com/ Ade

    Wow! I just started my blog in June and I am just learning about Reddit. I found this post because I was surprised to see the about 100 new visitors to my blog which had way less visitors the previous day! Needless to say I am amazed at the massive amount of traffic that Reddit can send to one’s website. Thanks for sharing!

  • http://newbostoncreative.com Shawn

    Were you able to move your IP address to the load balancer or did you have to get a new one and make some DNS adjustments as well?

  • Sean

    What size servers did you use? Would love to know your setup.

  • murray

    Helpful article. I’m going to install varnish on my dev server and see how the install and config goes. Looks pretty easy. Can someone explain the diffs between APC, Varnish & memcached? Can all of these coexist?

  • http://www.hardwarepal.com/ Nikolas Nikolaou

    I personally hit the homepage of Digg on the 15th of this month I had no such issues as I was using WPengine. Not a single glitch or slowdown on the site with 3300 people at once. Alittle over 200K pageviews for that day on my end.

Racker Powered
©2014 Rackspace, US Inc.