Web performance monitoring is the act of measuring the performance of a given set of metrics to see how fast a website or application is presented to an end user. Metric data is usually gathered via the two main types of performance monitoring: active synthetic monitoring and passive real user monitoring (“RUM”). A web performance monitoring tool takes the raw data that the metrics provide on your website or web applications, collates it, and presents it in a variety of ways so that performance issues can be effectively analyzed and quickly responded to.
The majority of end user response time occurs on the front end, so website performance monitoring is accordingly focused there. This stands in contrast to Application Performance Monitoring (APM), which traces a transaction’s performance through its back end services.
Let’s take a deeper look at web performance metrics and web performance analytics.
Web performance metrics are generally discussed as web/IT metrics. The technology stack and architecture used to present a site or application to an end user will provide efficacy support for determining which metrics are most important for that specific use case. Web performance metrics typically include page load times, DNS times, server times, response times, file size, and others. When discussed as part of a business use case, such as web performance optimization, the most relevant metrics may include shopping cart size, conversion rates, time on page, and number of page views. Here are the top ten web performance metrics to measure when focusing your overall web optimization efforts.
"Web performance metrics typically include page load times, DNS times, server times, response times, file size, and others. When discussed as part of a business use case, such as web performance optimization, the most relevant metrics may include shopping cart size, conversion rates, time on page, and number of page views."
Regardless of which web performance metrics you observe and collect, analyzing the data they provide consists of two key components:
When trending web performance data over time, RUM data typically follows a 24-hour end user “wake up and go to sleep” pattern because it is tracking real user traffic and “follows the sun”. Synthetic data, by contrast, will present as a smoother line because the condition of the emulated agents does not change over each 24 hour period. There may be times when either the synthetic dataset versus the RUM dataset will trend in different directions. Therefore, it is important to use both Synthetic and RUM measurement types as part of your web performance monitoring program.
When trending web performance data over time, consider using one of two data aggregation calculations:
Here’s the above RUM data using both the average and median values:
When analyzing web performance data, it is important to look at the data as a composite to get a holistic view of how end users are experiencing your site. There are two core questions to ask:
The long-tail is important to consider in addition to the average performance time. One way of considering the long-tail is to see it as a useful way to tell you what percent of users are having a bad experience, versus having to guess at it. As opposed to representing only a subset of users, the long tail is a gradient of experiences of all users. Not unlike considering the distribution of wealth in the world, long tail analysis helps determine what portion of users would benefit from your precious web optimization efforts.
There are two important visuals that can help with your analysis:
In the above histogram, we can see that most web performance data is clustered around the 3 – 5 second set of page load time buckets. The highest occurrence of web performance data is the four second bucket (8,474 occurrences) and the second highest occurrence of web performance data is in the three second bucket (7,857 occurrences). In other words, when asked, “What is the average performance for our end users?”, this histogram suggests the answer is, “The average performance for our end users is around three to four seconds.” However, as we can see, there are occurrences of data in the ten, 11, and 12+ seconds set of buckets as well i.e. the long tail.
Put another way, the [arithmetic mean] average value of the above dataset is 4,710 ms and the median value of the above dataset is 4,053 ms. So while “the average web performance” is around three to four seconds, there is an entirely different set of web performance data comprising the long tail. Unfortunately, when using a histogram, the long tail is compressed and consequently more difficult to read than the CDF.
The above CDF is constructed using the exact same data set as for the frequency histogram.
For the above example, the 25th percentile is 3,297 ms. This means:
The true power of the CDF comes in its taper points (which is why this type of visual is sometimes referred to as a “hockey stick” chart). In this CDF, we can see the taper start around the 85th percentile with drastic increases for each subsequent percentile. Evidently, the CDF does a better job of visually conveying the Nth percentiles for the long tails.
The most important aspect of any web performance monitoring program is how fast your website or application is presented to an end user. When looking through this lens, web performance monitoring must include:
Time-based synthetic and RUM data will present different patterns. Synthetic data will present as a smoother line, where RUM data will present as an end user’s “wake up and go to sleep pattern”. When using frequency histograms or CDF, synthetic and RUM data will present in the same general patterns; for example, a frequency histogram for both data sets will have a right skew “long tail”.
When it comes to web performance, there is no one-size-fits-all number. As long as you start with your end users in mind and work backward from there, your web performance programs will provide value for you and your business.
Develop a killer DEM strategy.
Our one-page checklist will help you determine your monitoring strategy and data analysis essentials.
“Catchpoint has been singularly instrumental for working out how we can release time from me, my team and others to focus on delivering value to the business because we’re taking less time to troubleshoot issues and concerns. Without Catchpoint, I would need to add at least another 3-4 resources to manage all the work that Catchpoint does today.”Learn how Thryv uses Catchpoint to monitor its web, application and service properties.