As internet traffic hits all-time highs we should all be thinking about how a spike in traffic to our web properties will be handled.
Whether it’s due to the pandemic or we simply need to get ready for a big online promotion, we should never forge ahead without knowing what our threshold for traffic is.
Performance testing is more nuanced and complex than simply throwing a bunch of traffic at your homepage and seeing how many visitors it takes to break the site. A proper testing strategy must be developed with all the necessary stakeholders.
How does a performance testing strategy look like? I’m glad you asked. Let’s break down all of the considerations you need to make when preparing for a performance test.
Here is what you should consider when deciding what sections/pages of your site to test:
- What is your normal web traffic, and how much traffic can you realistically anticipate based on some outside factor (marketing, press, pandemic, etc. )?
- What are the areas of the website you expect to be visited/utilized during the spike...and what other areas of the site are logically connected as a result? For example, a product landing page for business casual clothes or home haircut kits would then drive a user to the product detail pages for polo shirts or Flowbees.
- What are the most ‘vulnerable’ pages or sections of the site? Some things we usually consider are listed below. Usually, your development team can help you with these.
- Dynamic pages created in realtime that cannot be cached. An example may be a website search results page. Other examples would be user-specific pages, like Account or Shopping Cart.
- Pages that have a lot of conditional logic, spaghetti code or technical debt. Again, your development team surely knows where the bodies are buried.
Once you’ve identified your high traffic pages and your more vulnerable pages, you can prioritize your testing in this order:
- Start with the pages that are both high traffic and vulnerable.
- Next, test the pages you’ve categorized as vulnerable (but not high traffic)
- Last, test the pages you identified as high traffic (but not vulnerable)
Keep in mind that you may need to script multiple-page scenarios, like a user adding products to their cart and checking out. Tools like Blazemeter (built on top of JMeter) or LoadNinja help with creating automated scripts for that.
Now that you know what pages to focus on, you’ll want to plan to test the pages. You will either want to test on the live site or a staging site.
If you choose the live site you’ll want to be sure to do it after hours (non-peak times) and have the tech team on standby in case something goes wrong. It’s advantageous to use a live environment and experience exactly what your users will experience. However, you’ll risk taking down your live site and losing revenue while it’s down.
One tactic we’ve used in the past with load-balanced environments is to take one web server out of rotation and test it in isolation. You’ll want to be sure all the traffic doesn’t go to the same DB server though, creating a bottleneck and impacting real users. If you choose to test a staging server, do your best to test a real-world scenario.
Here are some questions to ask the team when setting up a staging/test site:
- Is the test server a good replica of the live site (same virtual memory, etc.)?
- Do we have the same architecture in staging as we do in production? For example, if there is a WAF or a cache in production, do we have those set up in staging too?
Your testing could yield false positives and false negatives if your staging site is not as close a copy as possible of your production environment.
Once you’ve established what pages to test and where to test you need to determine how much traffic to throw at it. After all, if the most traffic you expect on the best day ever, is a few thousand visits, there is no sense going overboard. And vice-versa, you don’t want to miss the opportunity to test for your most heavy traffic spikes.
We like to say you want to design the church for Easter Sunday. We use Blazemeter for performance testing, but there are many others. Here is a great article on how to determine your virtual user count:
https://www.blazemeter.com/blog/8-tips-decide-number-concurrent-users-your-test
Once you’ve determined your user counts, you’re ready to create your tests.
Keep in mind that you need to test a site from 3 different perspectives.
- The response from the server. This is the most obvious and easiest to implement. You just point your tool at the pages you want to test and it will tell you how long it takes the server to respond. This does not tell the whole story though...
- The load on the server. While the test is being performed and you are throwing your thousands of requests at the server you’ll want to make sure you are recording what’s going on on the server. Be sure to use a tool like Perfmon, NewRelic, or similar to see what breaks. Be sure to include the database server or app service as well.
- The ‘client-side’. The responses from the server may be coming back wicked-fast, but the site may still feel like watching paint dry. This is because once the server sends a response, your browser still needs to render all the javascript, HTML and even make any additional calls to outside web services. So, I recommend using a tool like Google Page Speed Insights to be sure that you are testing what the user is actually seeing!
**One important note: The issues flagged by these tools may or may not be caused by load (for example, it could be a runaway javascript). So, be sure to test with/without load on the site.
Now you are ready to test your site.
A few tips before rollout:
- Remember to start small to be sure you don’t bring down the environment.
- Be sure to check your hosting provider’s rules for performance testing. Here they are for AWS.
- Check which tools are available for your specific environment. For example, for Azure.
Now you have a roadmap for ensuring that your site is ready for a crush. The actual tools and implementation will depend on your technical environment, but you’ll want to be sure to cover all the bases outlined here.
Go crush your performance testing before your site is crushed by traffic.
