Performance-boosting techniques for a high traffic Drupal site

Last week we were thrilled to learn that a site we developed for the North Carolina High School Athletic Association (NCHSAA) was selected as Best Sports Website in the 2014 Blue Drop Awards.

Over the coming weeks we'll write a few posts highlighting some of the features of the site. In this post, I want to focus on some of the performance techniques we used to ensure quick page loads.

Score ticker

One of the key requirements for the site redesign was to implement a score ticker based on a feed of data from NCHSAA-affiliated schools. Although I encourage you to visit nchsaa.org to interact with the score ticker (Frank did an outstanding job writing it in jQuery and making it adapt well to mobile devices), here is a screenshot showing what it looks like:

The feed of data is generated by MaxPreps, a site which crowd-sources data from athletic events.

The problem is that the data retrieved from MaxPreps comes in at a few megabytes; much more than we would want to serve to the client when they are visiting the homepage, especially if they are coming from a mobile device. The feed data is so large because there can be multiple sports in the ticker, with multiple divisions, and each division can have dozens or hundreds of games.

To deal with this issue, we did a few things:

1. On page load, we check a custom cache bin cache_nchsaa_banners to see if we have data for each sport that we want to display in the ticker. Each row in the cache bin corresponds to a valid sport, e.g. boys_basketball or girls_soccer, etc. (As an aside, Lullabot's guide to caching is an invaluable resource to those new to Drupal and caching.)
   • (a) If there is no data in the cache, then we call out to the MaxPreps API. MaxPreps might send hundreds of games for a particular sport, and a large number of these may belong to schools that are not affiliated with NCHSAA, so we make a further optimization by removing all of these entries.
   • (b) Once we've pared down the data, we run it through a function that generates the HTML that we'll output into the page; that HTML is then cached.
   • (c) A div is output to the page, <div id="banners-ajax"></div>.
2. Next, our client-side code kicks in.
   • (a) On page load, the client's browser makes a call to /api/v1/banners/onload/json and downloads about 8KB of data that populates the visible part (#banners-ajax) of the score ticker block: the names of the sports that have data, the divisions for the first sport, and the games for the first division loaded for the first sport. So in the screenshot, the labels "Baseball", "Soccer (M)" and "Class 1A - 4A" are loaded, along with the game data for Class 1A. But no data is downloaded for the other sports or divisions.
   • (b) From there, when a user clicks on other sports or divisions, API calls are made from the client-side code to URLs like /api/v1/banners/soccer/json or /api/v1/banners/soccer/division-1a/json. In this way, we limit the amount of data requested from the client and served by the site to just what the user has requested.

The trade-off with this approach is that there is a small delay as the user clicks from one sport to the next; but for us this was far preferable to the alternative of sending hundreds of kilobytes or 1-2 MB to the client.

Sports menu

Another performance optimization we made was under the "Sports" menu. Each sport menu item contains some "fast facts" about the sport (start and end of season, play-off dates, etc). All of this is managed in each sport node in a specific field. To load the data, we JSON-encoded an array of sport names and their fast facts, then output it at the top of the page (this comes out to around 24 KB).

This data is cached, so it loads very quickly. A further optimization would be to load it only once the user hovers over the Sports menu, but we did not implement that just yet. Below you can see the code that is used to check the cache, or load data fresh from the database.

Varnish and Memcache

The site overall benefits tremendously from having a Varnish cache sit in front of it. What is Varnish?

Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. (source)

Since there are relatively few authenticated users (authenticated user caching is a very different set of problems, and for that I recommend you read "Authenticated User Caching Concepts in Drupal 7"), most of the site's content is served directly from the Varnish cache, instead of requiring time and resource-consuming database queries.

Memcache is also enabled in production, along with Entity Cache. This further reduces the load on the database server and makes retrieving content quicker.

Kudos to Acquia Cloud for making it incredibly simple to enable and configure both of these services.

Opt for simpler implementations instead of module "open buffet"

Wherever possible, we opted to create simpler implementations rather than adding another module, and thus tried to avoid the "open buffet" problem that we see frequently in Drupal sites.

As an example, one of the requirements of the site was to implement an ad tracking service. Of course, an ad tracking service could be something very simple or something very complex. In the project planning process we determined that what was really needed was a way to report metrics to NCHSAA's sponsors on how many people had clicked on a sponsor's banner ad.

Since we were already using Google Analytics, it was straightforward to set up an Advertisement content type with a field for uploading an image, and then use a theme template to generate Google Analytics tracking code for each Advertisement node. No additional modules, and as a bonus we were able to re-use the functionality offered by an existing module (Google Analytics).

Develop with an eye towards performance

The examples above are indicative of the overall approach we took when building the site: simpler is better, avoid adding modules if you don't really need them, and load data only when the user asks for it.

This quick overview of some of the performance techniques we used on the NCHSAA aims to spark ideas while highlighting aspects of the NCHSAA project. Let me know if you have any questions or suggestions for how to do things differently.