In early May, the performance team deployed a code change that reduced our start render times by over 40%. While we have been making gains on all of our verticals with system-wide changes, this was one of the first deploys that targeted a specific vertical. The change itself was fairly straight-forward, but the process we went through is more indicative of much of the performance work we have done here at Vox.
Identifying the problem
Every optimization starts with looking at one or more of our sites, and identifying a performance blocker that we can address in a meaningful way. We have constant monitoring of all of our sites, but more commonly these blockers are found by looking directly at individual pages that are frequently trafficked. We spend a good amount of our time staring at data coming from Web Page Test and Chrome’s developer tools.
This process of identifying script usage started many months ago. Each script was analyzed and all dependencies and functions were documented, which allowed us to divvy up the files into the aforementioned groups and properly understand their purpose.
Updating the codebase
Once the scripts we loaded in the head were documented and categorized, we began the process of refactoring them in a more forward-looking fashion. This primarily meant starting from scratch with how we work with advertisements on the front end. While we had great latitude with how the scripts were refactored, we felt it was best to keep how we were calling advertisements to remain the same. To prevent any outages or advertisement-related issues, we only optimized how the scripts themselves worked.
We removed cruft. We removed functions that could safely be moved to the backend. We optimized the amount of data that was being sent to the front end to be as minimal as possible. We questioned assumptions. We inlined scripts that were vital to our page load, and removed everything else. We worked with several other engineering teams to ensure that our proposed changes would work well with future development on the platform.
The results looked promising, and certainly removed the bulk of our render blocking scripts. We ensured that the alterations would remain future-friendly, at least within the context of current development thoughts.
QA and testing
Here at Vox, we’re fortunate to have an amazing QA team that can confirm all automated tests, as well as ensure interactions that may not be completely covered within automation are working as expected. Their hard work discovered two small bugs, both of which were simple to fix. Both were the result of poor code written in the past, and minor updates fixed them without issue.
Although slow and aggravating, since it was outside of our hands for some time, this was one of the more critical pieces of our process. With 8 verticals all working off a singular codebase, sometimes automation wasn’t enough. And while we can usually catch most minor issues during development, performance changes generally have wider implications than single features. Fixing one-off issues and removing bits of antiquated code is important to ensure full support for everybody.
Deployment and confirmation
We’ve refactored the code, reviewed it, QA’d it, and now it’s time to schedule the deploy. Because this will affect advertisements, and therefore revenue, we wanted to deploy it first thing on a Monday, which would give us the best reporting possible on any potential revenue impact. Working with engineering and our revenue teams, we ensured that any and all worries were allayed. We also ensured that we ran a full battery of performance tests before and after the deployment to confirm the desired effect.
The results were wonderful, cutting down our start render time by over 40%. The video above is simulating a 3G connection, where we have a very marked improvement. It has been one of the single largest optimizations to date, and worked precisely because of the interaction between the Performance Team and other engineers, QA, and revenue teams.
The future and thoughts
This entire process is a good of example of how we’ve been approaching performance at Vox Media. We find bottle necks, we make progressive changes, we test, we repeat. The work of the performance team is never done, it just continues to optimize the product more. As we’ve said in the past, performance isn’t a destination, it’s a journey.