Black Friday Assurance—delivering performance as promised

It’s that time of year again. The Thanksgiving week ushered in the year’s biggest sales event, Black Friday and Cyber Monday. Retailers stuck to the tried and tested formula, most websites went live with deals days ahead, unveiling bigger discounts leading up to Black Friday. The deals and offers were not limited to Black Friday but continued through the weekend and Cyber Monday.

Companies prepare for the holiday season building compelling marketing strategies, optimizing all applications and implementing an effective monitoring plan to ensure there are no outages or performance issues during the sale period.

Over the years, we have noticed that although retailers put in extra effort around monitoring their applications, there is always a gap in the monitoring strategy. The monitoring strategy is entirely focussed on handling complete outage or site reachability. This leaves the retailers and service providers vulnerable to other performance issues.

This year, Catchpoint wanted to address this obvious-yet-overlooked gap in most monitoring strategies with our Black Friday Assurance program. We launched the assurance program to help customers make the best of the holiday season without worrying about application downtime or sluggish performance.

Monitoring strategies evolve—an ongoing process that cannot be limited to a certain time period. It is crucial to have the right tests set up and alerts configured so you get data that is relevant and highlights specific performance trends. In this blog, we discuss how we, Catchpoint, helped our customers during the holiday sale.

What did the Assurance program involve?

Customers who opted for the Black Friday Assurance program had access to our expert team of performance engineers responsible for setting up tests and monitoring their applications 24/7. The program delivered performance protection for end users during the weeks leading up to the most important shopping days of the year.

Catchpoint’s Black Friday Assurance program aimed to:

• Help our customers configure the best testing strategy prior to the Black Friday.
• Quickly detect, troubleshoot, and fix outages during the Black Friday and Cyber Monday week.
• Analyze application performance after the event and benchmark against competitors in the industry.

Prepping for the sale

The Catchpoint Black Friday Assurance team started preparing weeks in advance, setting up different tests, alerts, and dashboards. Our strategy included:

1. Setting up multiple test types

• End-to-end transactions to ensure users are interacting and navigating, adding validations on each step, randomizing product search criteria.
• DNS tests
• Uptime tests

2. Configuring different features

• Monitoring third-party impact, CDNs, API integrations, etc.

3. Configuring relevant alerts

• Tracking performance drops/performance alerts
• User experience alerts
• Page composition alerts
• Availability and reachability alerts
• Content owner alerts – is the content provided by a first or third party

4. Setting up dashboards

• Multiple dashboards were set up to quickly identify performance degradation and outages

Delivering the Commitment

Catchpoint committed to assuring a fast and positive end-user experience for customers who opted in for the Back Friday Assurance program. We promised to detect and optimize some of the major performance issues faced by ecommerce sites and other service providers including:

Our Black Friday Assurance team started monitoring websites 24/7 two weeks prior to the sale event. The team identified multiple issues during the monitoring period. These incidents were caught real-time and accounted for approximately 97 hrs of total outage across 150 websites. Such performance issues add to user frustration resulting in negative end-user experience.

Catchpoint was able to detect numerous application and delivery issues during Black Friday through Cyber Monday. We have listed a few of these below.

• Cart and checkout issues
• Latency due to third-party tags
• Website outage
• Connection failures
• Search result page not working
• Sign in/login page performance drop
• Product page error – out of stock

The image above illustrates a transaction process for one of the websites we monitored and it shows the performance degradation in the 4th step which corresponds to the “add-to-cart” action.

We identified the Coremetrics outage that caused hundreds of websites to significantly slow, including retail giants like Macys and Kohls. Our customers were immediately notified so they could address the issue quickly and mitigate the impact it had on end-user experience.

Read the detailed outage analysis here.

The outage that impacted Cloudfront was another incident we identified a few days before Thanksgiving.

Catchpoint helped prep customers to manage any performance incident quickly and efficiently—helping hundreds of our customers handle performance issues and deliver better end-user experience. The proactive strategy was key to delivering what we committed to with the Black Friday Assurance program.

Takeaways

The holiday season pushes every retailer to rethink their marketing strategies to drive more users to their online/offline stores. And every year, we see major brand websites brought down by performance issues, issues that the right monitoring strategy would resolve faster. The performance trends during the sale season continue to support the fact that performance does have a direct impact on business.

With the Black Friday Assurance program, Catchpoint is helping customers refocus on three important aspects of monitoring:

• Preparation matters: Don’t limit application performance monitoring to the holiday season. It is an ongoing process, the data captured helps you understand what to optimize and why you may not be engaging enough customers. Prepare in advance, analyze historic trends, invest time in A/B testing and optimize processes.
• Testing strategy: A monitoring strategy is not about setting up hundreds of tests. It only creates unwanted complexities and makes monitoring overwhelming and less effective. You need to run tests targeting critical transactions and pages, so you are analyzing only the data that matters.
• Alerting strategy: You should maintain alerts on a similar line. There is no need to alert every data anomaly. Configure alerts for trends that are relevant to the functioning on the application and those that impact end-user experience.

A monitoring strategy built around these points is all you need to handle performance issues year around.