Black Friday Assurance: Elevating End-User Experience Monitoring

The holiday sale season is a busy time for both Catchpoint and our customers; keeping track of performance in the eCommerce industry and helping our customers avoid major outages so they can keep their business targets on track is a 24/7 task.

Catchpoint’s Black Friday Assurance program, now in its second year, served hundreds of major retailers. The thanksgiving week puts all types of monitoring strategies to test and the performance data from this period provide interesting insights. In this blog, we look at some of the performance issues that the Black Friday Assurance team detected over the last two weeks.

From November 22 to December 6, the Black Friday Assurance team set up, managed, and monitored websites and applications for customers who had signed up for the program. The monitoring added another layer of performance protection for end users during the holiday sale season.

Last year, we successfully guided over 100 customers through Black Friday, managing and optimizing for some of the most challenging performance issues eCommerce sites face, including:

• Performance issues and bottlenecks due to latency introduced by CDN or DNS
• Transaction failures including add-to-cart and payment gateway issues
• Script and other functionality errors that impacted end users

And this year we detected a range of issues such as:

• Poor page structure impacting user experience
• HTTP 503s and 403s
• Slow page load times
• Users redirected to waiting/queue page as traffic multiplied
• Caching issues – high Cache miss rate
• Unoptimized third-party content
• Capacity issues at origin
• Connection issues – unreachable hosts
• Volatile requests in the page
• CDN mapping issues
• Product pages slow down seen across multiple sites
• Image optimization failing in certain regions
• Slow search results
• Add-to-cart failures returning 503s
• Payment gateway failures
• Shopping cart crashes

Let’s look at some of these in detail.

Slow Performance: We noticed slow response time for multiple pages of one of our customer’s website due to high wait time (time taken by the server to respond to the client request with the first packet of information). The graph below shows the impact on the product, add-to-cart, and cart pages.

Performance volatility: We noticed volatile performance on a popular eCommerce site, mostly related to contents loading from a specific host, a specific cloud provider.

Image optimization issue: A major online retailer had recurring slowdown at specific intervals during the week. We were able to narrow down the cause; the website was using real-time image optimization, which was occasionally slow resulting in high wait times.

Third-party issue: Another eCommerce site faced intermittent performance degradation that impacted the end-user experience on the website. There was an increase in connect time for certain third-party hosts. The chart below illustrates the periodic spike in webpage response times as a result of the high connect time.

Preparation is KEY

We believe that preparation is key to making it through the Black Friday / Cyber Monday week without a hitch, and the Black Friday Assurance ensured just that. We were able to ensure that our customers were prepared to handle incidents and quickly resolve issues.

There is a lot that goes into ensuring you are prepared. Testing all new features and functionality before it rolls out on the website and implementing a comprehensive monitoring strategy is just one of the many things your organization should prioritize when prepping for the big sale. Here are some important steps you should consider as a part of this exercise:

1‍. Optimize holistically

End-user experience monitoring is not limited to measuring uptime or downtime. A holistic approach is necessary to understand the true end-user perspective. There is a lot of complexity that powers a single application, from the multiple components involved to the multiple third-party services it relies on, so it is vital to analyze and optimize the performance of every component.

2. Monitor critical user journey

The search, product, cart, and checkout processes are critical to the functioning of any eCommerce website. Performance issues at any point in the user journey can impact the end-user experience, so it is important to monitor these processes/pages and to include critical user journeys in the application optimization strategy.

3. Map each component in the delivery chain

Make sure you know the third-party vendors that your organization relies on for delivering service to your customers—Content Delivery Networks (CDNs), Transit and Internet Service Providers (ISPs), Wireless Networks, User Browsers, etc. For mission-critical services hosted beyond your own firewall, you must identify visibility gaps and blind spots.

4. Capture availability, reachability, performance, and reliability data

To regain end-to-end visibility of your application, your monitoring strategy must capture data beyond just uptime; it must capture data for all four pillars of digital experience monitoring: availability, reachability, performance, and reliability.

**5. Don’t rely on averages during peak traffic

**Every digital experience matters, which is why you can’t rely on averages when monitoring end-user experience. When monitoring performance, averages do not provide an accurate view into performance for tail-end users for whom services may be much slower based on conditions specific to their geographical region. Instead, rely on percentile median value for a more accurate view of performance to pinpoint issues for all users.

6. Correlate Technological and Business KPIs

You should be able to correlate the business value of website optimizations and improvements. These metrics matter because they determine what and where to optimize, which in turn determines visibility into your website and end-user experience.

In addition to these steps, you need to monitor continuously and set baselines for your key digital experience metrics, as well as compare your applications and services to those of your competitors. An ideal monitoring strategy should include both synthetic and real user monitoring (RUM), enabling you to compare, evaluate, and improve your service. You will then gain visibility into every key aspect of the application delivery chain. Preparation keeps you ahead of the game, and you are ready to handle any performance crisis with minimum impact to end-user experience.