Reducing the start render time on Chorus

In early May, the performance team deployed a code change that reduced our start render times by over 40%. While we have been making gains on all of our verticals with system-wide changes, this was one of the first deploys that targeted a specific vertical. The change itself was fairly straight-forward, but the process we went through is more indicative of much of the performance work we have done here at Vox.

You can see the results yourself on Curbed right now. I have a spoiler alert for anybody curious on the technical change: We moved JavaScript to the footer. But technical changes are easy to document, so we wanted to share our process of how we go from identifying the issue all the way to deployment and optimization.

Identifying the problem

Every optimization starts with looking at one or more of our sites, and identifying a performance blocker that we can address in a meaningful way. We have constant monitoring of all of our sites, but more commonly these blockers are found by looking directly at individual pages that are frequently trafficked. We spend a good amount of our time staring at data coming from Web Page Test and Chrome’s developer tools.

In this instance, we have known for some time that our JavaScript in the head was causing a substantial delay in rendering. The problem goes deeper, as much of our JavaScript was written with the expectation that it would be in the head, or was a dependency of other scripts that needed to be within the head. For us to fully identify the problem, we had to look at all of the scripts we were loading. We divided them into scripts that could be moved without issue, scripts that needed re-writing, and scripts that one or two functions (e.g. browser tests) could be pulled from.

This process of identifying script usage started many months ago. Each script was analyzed and all dependencies and functions were documented, which allowed us to divvy up the files into the aforementioned groups and properly understand their purpose.

Updating the codebase

Once the scripts we loaded in the head were documented and categorized, we began the process of refactoring them in a more forward-looking fashion. This primarily meant starting from scratch with how we work with advertisements on the front end. While we had great latitude with how the scripts were refactored, we felt it was best to keep how we were calling advertisements to remain the same. To prevent any outages or advertisement-related issues, we only optimized how the scripts themselves worked.

We removed cruft. We removed functions that could safely be moved to the backend. We optimized the amount of data that was being sent to the front end to be as minimal as possible. We questioned assumptions. We inlined scripts that were vital to our page load, and removed everything else. We worked with several other engineering teams to ensure that our proposed changes would work well with future development on the platform.

The results looked promising, and certainly removed the bulk of our render blocking scripts. We ensured that the alterations would remain future-friendly, at least within the context of current development thoughts.

QA and testing

Here at Vox, we’re fortunate to have an amazing QA team that can confirm all automated tests, as well as ensure interactions that may not be completely covered within automation are working as expected. Their hard work discovered two small bugs, both of which were simple to fix. Both were the result of poor code written in the past, and minor updates fixed them without issue.

Although slow and aggravating, since it was outside of our hands for some time, this was one of the more critical pieces of our process. With 8 verticals all working off a singular codebase, sometimes automation wasn’t enough. And while we can usually catch most minor issues during development, performance changes generally have wider implications than single features. Fixing one-off issues and removing bits of antiquated code is important to ensure full support for everybody.

Deployment and confirmation

We’ve refactored the code, reviewed it, QA’d it, and now it’s time to schedule the deploy. Because this will affect advertisements, and therefore revenue, we wanted to deploy it first thing on a Monday, which would give us the best reporting possible on any potential revenue impact. Working with engineering and our revenue teams, we ensured that any and all worries were allayed. We also ensured that we ran a full battery of performance tests before and after the deployment to confirm the desired effect.

The results were wonderful, cutting down our start render time by over 40%. The video above is simulating a 3G connection, where we have a very marked improvement. It has been one of the single largest optimizations to date, and worked precisely because of the interaction between the Performance Team and other engineers, QA, and revenue teams.

The future and thoughts

We are in the process of moving all of our verticals to a similar system that will future-proof how we render ads and work with javascript in general. Much of this process will either be repeated for each site, or at the very least changes will be confirmed in similar ways.

I am not entirely certain that inlining our advertising JavaScript is the perfect answer for our performance concerns, so I’ll continue looking for better ways of working with our advertisements. Even though this was a large fix, we are constantly looking at better ways of addressing our performance concerns.

This entire process is a good of example of how we’ve been approaching performance at Vox Media. We find bottle necks, we make progressive changes, we test, we repeat. The work of the performance team is never done, it just continues to optimize the product more. As we’ve said in the past, performance isn’t a destination, it’s a journey.

If you want to learn more about performance at Vox Media, Ian Carrico and Jason Ormand will be presenting at Velocity Santa Clara on Wednesday, June 22nd.