If you did your holiday shopping online this year, you’re not alone. Cyber 5, the five days between Thanksgiving and Cyber Monday, represented one-fifth of all eCommerce sales for November and December in 2021 (despite a slight decline in overall spending since last year).

Americans shelled out $8.9 billion on Black Friday deals and $10.7 billion on Cyber Monday specials. As such, Cyber 5 represents a critical time for companies to keep their websites up and running smoothly and able to handle the increased traffic load.

If you want to know how the top eCommerce sites performed overall during Cyber 5, read on.

Cyber 5 2021 Benchmarking Report

Every year for the last decade, as part of our Holiday Performance Monitoring Assessment, Catchpoint has created a Benchmark Report that tracks the digital performance of the world’s major retailers across Cyber 5.

Our latest Report included Amazon, Alibaba, and Walmart. The results reflect the continuous pressure eCommerce and online retailers feel to deliver outstanding customer experience to their end users. This pressure is never felt more intensely than during the busiest period of the year.

It’s worth remembering that the end user experience is reflective of the entire journey traveled across every service provider involved in the delivery of content.

‍_“When consumers shop online, they travel from their last-mile local ISP, and crisscross a vast web of interconnected long-haul carriers with peering relationships that might not be friendly for some. They're handed off to any number of content delivery networks and DNS providers until they finally reach the retailer's website. But their digital journey doesn't stop there. Behind the scenes and under virtual covers, consumer browsers and apps interact with the retailer's backend servers, network equipment, databases, data analytics, and third-party services.”_ Karthik Suresh, Manager of Professional Services, Catchpoint

Catchpoint’s benchmarking tests on 32 major retailers ran from 12:01AM EST November 25 until 11:59PM EST November 29, 2021. The tests measured critical transactions on the selected retailers’ home pages and product details pages, as well as their search results. The tests ran at five-minute intervals, with measurements calculated by median value.

The overall goal of observing webpage performance is to compare how U.S. eCommerce companies handled traffic spikes during the most concentrated shopping period of the year.

Critical performance metrics measured

Metrics measured during the days of the 2021 Cyber 5 included: ‍

DNS - The time it took the Catchpoint node to resolve the DNS for the primary URL of the test(s). Industry Standard: <50ms ‍

WAIT - The time from when the connection was established successfully and the request was sent to the server, to the time we receive the first byte of response for the Primary URL of the test(s). Industry Standard: 100ms - 200ms

RESPONSE - The time it took from the request being issued to the primary host server responding with the last byte of the primary URL of the test(s). Industry Standard: <300ms

LARGEST CONTENTFUL PAINT - Largest Contentful Paint (LCP) is an important, user-centric metric which marks the point in the page load timeline when the page's main content has loaded. Industry Standard: Good <2.5 sec / Needs Improvement 2.5 sec – 4 sec / Poor > 4 sec

WEBPAGE RESPONSE - The time it took from the request being issued to receiving the last Byte of the final element on the page. Industry Standard: 3 sec – 4 sec

Critical performance metrics measured during Cyber 5 (Catchpoint)

Let’s take a look at the Cyber 5 2021 Report and some of our key findings.

Cyber Monday performance challenges

While most eCommerce and digital retailers were well prepared in advance of Cyber 5, many still experienced challenges when traffic volume spiked.

On the day of Cyber Monday, for instance, the busiest shopping day of the year, Bed Bath & Beyond rendered content 500ms slower when compared to before and during Black Friday.

LCP for the home retail store on Cyber Monday measured at 1896ms compared to 1352ms before Black Friday and 1423ms on Black Friday itself.

Indeed, we saw a increase in Webpage Response on Cyber Monday for many top sites, including Toys R Us, Macy’s, and Nordstrom.

Page size matters

While page size clearly made a difference to web performance for some companies (the page size of Bed Bath & Beyond, for instance, increased by 1MB on Cyber Monday), for others, it posed fewer problems.  Pepsi and Apple topped the benchmarking list at 1.1 MB each. Their Webpage Response times, however, were notably different. Pepsi took six seconds to load its home page compared to under 1.5 seconds for Apple.

Webpage response is affected by the number of hosts

Apple topped the list on Webpage Response across Cyber 5 by loading its home page within 1.5 seconds every day. Amazon ranked second with the delta between it and Apple at around 1.5 seconds.

Remember the industry standard for loading the home page is 3-4 seconds. Only Apple and Amazon managed to meet this standard and load the page within four seconds. Every other retailer whose web performance we observed experienced Webpage Response times between 4-9 seconds.

‍Was Apple’s impressive performance largely due to the fact it only had three hosts serving content on the page? Most likely so, since when we correlate our data, we can see that the sites with the largest number of hosts had the slowest performance across Cyber 5.

Network issues can impact page rendering and user experience

As shown below, Calvin Klein intermittently experienced problems in its Response Time across the five days.

If we focus on one instance of high Response time, as seen in the screenshot below, we can see that a high DNS resolution time (3923ms) had a ripple effect on the other web metrics. This impacted the LCP, which tells you when the largest piece of content was rendered on the page.

There was a delay in LCP at 4,460 (compared to Calvin Klein’s average over Cyber 5 of 1.5-1.6 seconds) and a delay in Webpage Response time (15227ms) due to the high DNS resolution time.

Your user’s first digital experience with your brand is through DNS. By monitoring for DNS performance, you can pinpoint when there are issues and why, and identify problems with the DNS provider.

Some major retailers struggling with high wait times for search results

While Bed Bath & Beyond topped the list in terms of wait time for its Search Results page, it wasn’t alone. Bath & Body Works, Nike, and Gap all had wait times greater than 600ms. Remember, the industry standard is 100–200ms.

Office Depot saw its wait time slow by 50ms on Cyber Monday. However, this was the least of Office Depot’s woes that day, since its entire site went down at 11:20AM EST on Cyber Monday, staying offline until 2:08PM EST. The retailer had to publicly apologize and decided to extend its Cyber Monday sale through Tuesday to assuage the frustration of customers.

Looking ahead to 2022: the big takeaways for eCommerce and Retailers

As we enter 2022, we must acknowledge that doing business online has become the norm. The pandemic has only sped up what was already inevitable. Online shopping, SaaS tools, remote work – we all depend on the Internet for nearly everything we do.

The big takeaways from Cyber 5 emphasize this point, showing us that:

Preparation is queen

Despite Office Depot’s near three hours of down time, compared to 2020, there were fewer major outages across Cyber 5 this year. Given how riddled the entire last year was with major outages (from Salesforce to Facebook), why might that be the case?

It’s likely that many network operators ensured that site maintenance happened before holiday season, meaning fewer outages since the maintenance work had already been performed.

In terms of web performance (and “slow time is the new downtime”), many retailers appear to be putting in the work to improve crucial metrics such as Wait Time, Largest Contentful Paint, and Webpage Response.

We can’t stress enough what a critical effort this is, with studies showing that digital companies lose customers after just two seconds of slow loading time. Check out this study from Core Web Vitals on the intrinsic link between web perf and business impact.

The work of developers to improve web perf can powerfully pay off at the busiest times of the year. One of this year’s biggest winners was Apple, who as we’ve seen, put in the work on their core metrics.

Optimization must be performed consistently

To remain up and running, let alone competitive, you need to constantly be optimizing your site for the best performance. Here are some optimization tips you should consider as part of your strategy, in particular for preparation for periods of high load:

• Optimize holistically – A holistic approach is crucial to understanding the true end-user perspective. End user experience monitoring takes into account the complexity involved in powering a single application.
• Map every component in the delivery chain – Take the time to understand which third-party vendors are part of delivering service to your customers. From CDNs to user browsers, you need to monitor all the mission-critical services hosted beyond your firewall.
• Don’t use averages during peak traffic – Averages don’t provide an accurate enough view into performance for tail-end users for whom services may be significantly slower due to geography. Instead, use percentile median value for a more precise view of performance to detect issues for all users.

Observability is necessary year-round

Given how the holiday shopping season is increasingly spreading out, performance management increasingly requires year-long proactive observability.

One of the most important things you can do to reduce latency and prevent outages is to institute a robust observability strategy. Run active observability testing to replicate extreme load spikes for differing kinds of load (mobile, desktop) as often as possible.

These tests allow you to performance benchmark, baseline, and understand the flow of network traffic.  Once you have that information, you can prepare accordingly, for instance by distributing increased traffic load across servers in advance of expected traffic spikes.

And finally…Is Cyber 5 dead?

While Cyber 5 was enormously profitable this year for many retailers, it’s no secret that the deals started earlier than usual.

"With early deals in October, consumers were not waiting around for discounts on big shopping days like Cyber Monday and Black Friday," said Taylor Schreiner, director of Adobe Digital Insights.

As we have become more accustomed to shopping online, the holiday season will spread out yet more. Check out our 2022 eCommerce Holiday Calendar to know when you need to scale your infrastructure to meet upcoming demand.