How to include multiple perspectives in your monitoring strategy

The internet is a complex place. Problems can be hard to identify, and we don’t always know where the roots lie. You’ve likely invested time and money building a monitoring strategy.  The strategy likley includes purchasing or building monitoring tools to help you see when and where problems occur and determine whether they’re within or outside your control.

• Is your service provider experiencing an outage?
• Are there network routing issues?
• Are regional ISP issues impacting how your end users experience your application?
• Are there problems with 3rd party services you rely on, such as Content Distribution Networks, DNS providers or Ad Servers?

We all know by now that how customers and end-users perceive your application is critical; the customer’s experience is the product. That is the main investment justification for deploying monitoring systems. You need to be able to see performance from the end user’s perspective. Performance from a mobile device will be very different than performance from a major broadband ISP or the cloud.

The IT landscape, from technology to people organization (SRE), to cloud migration, is also now being altered by customer demands. An end user doesn’t know whether it is their service provider, a routing issue, or the application they are trying to access that is having problems. If something breaks or is not performing as expected, chances are, users will get frustrated with the application.

Include multiple perspectives in your monitoring strategy

Perspective makes a difference. Consider the fable of the five blind men and an elephant. Each person examined one part of the elephant’s body, like the tail or the tusk. As you can imagine the descriptions are completely different – the tail was “rope,” the tusk was a “spear.” The same thing happens when you are only monitoring from a single lens.

You don’t see the whole picture if your synthetic monitoring only occurs from the cloud, or if you’re not monitoring from wireless providers. Your understanding of your service performance and customer experience will inevitably be limited if it doesn’t include multiple perspectives.

A monitoring strategy should provide as much insight into the end-user experience as possible. Ask yourself the following questions when developing your monitoring strategy.

• Are my end-users’ internal employees or external customers?
• Which SaaS and internal applications are being used by our employees?
• How can we identify first mile problems that we cause vs. external last mile problems?
• How do we determine performance on mobile networks?
• What is the experience like for real users?
• Is it a 3rd party (CDN, DNS, APIs, Ads…) causing the issue?

You need to make sure your strategy and tools provide complete visibility of the end user’s experience along the service delivery chain they experience. Monitoring only from within the data center limits external visibility while monitoring only from consumer ISPs includes issues outside of your control. It is essential to have a mixture of vantage points for greater insight into performance.

Monitoring from backbone and broadband providers

The most consistent, most reliable source of active testing is from facilities directly connected to major, tier 1 Internet backbone and broadband providers, such as Level3, NTT, Comcast, British Telecom, China Telecom. These vantage points are in line with service delivery to your end users but don’t suffer the random noise often seen in last mile networks, such as shared bandwidth contention, mobile dead spots, and other issues on the local access link to external customers.

For this reason, a good monitoring strategy must include monitoring from regional backbone and broadband service providers.  These are excellent locations for baseline performance data collection and alerting. Backbone and broadband provider metrics can then be correlated with data from other vantage points, such as internal first mile, cloud, last mile fixed and mobile, to help verify and localize problems.

Monitoring from the cloud

As Internet services and infrastructure continue migrating to the cloud, monitoring from the cloud is of growing importance as well. Migrating to the cloud typically means moving or extending your data center to a public cloud provider, like AWS, Azure, Google, Tencent or Alibaba. In this case, monitoring cloud-based services from the same cloud provider and location is equivalent to first mile monitoring. You can compare these metrics with data from other vantage points to pinpoint problems.

Similarly, if you rely on cloud-to-cloud APIs or other inter-cloud or hybrid services, you’ll need performance data from your cloud providers to have performance visibility along the same delivery chain.

Be careful not to over-rely on cloud-based monitoring.  It’s not a replacement for monitoring from Internet backbone or broadband locations. Just because your services have migrated to cloud doesn’t mean your end users have moved there too. If your customers access your services (cloud-based or not) from local ISPs, then monitoring from the cloud is not collecting performance metrics on the service delivery chain that your customers experience. So, you won’t detect external Internet-based problems they commonly face. As such, a cloud-only monitoring strategy can leave you dangerously blind. http://blog.catchpoint.com/2018/03/23/synthetic-monitoring-cloud-vs-backbone-broadband-nodes/

Measuring performance from the first mile

It’s essential that monitoring tools notify you when performance and availability fall below acceptable levels. This often requires separating signal from noise. You need the cleanest information with the least amount of noise so that you can trust the alert. You don’t want to be paged at 2 a.m. from an outage caused by noise.

One way to eliminate the noise is to monitor from the first mile. If you host your application in the cloud, measure from the cloud. If you host your application in a data center, collect metrics from within the data center.

Monitoring from corporate locations

You need to monitor from where your users are accessing your applications. When answering the questions above, if you indicated you need to monitor internal applications for employees or SaaS applications then gather metrics from corporate locations where you’re using the applications.

This requires deploying nodes or agents in corporate offices, buses, or even airplanes. With data collected from internal vantage points, you see the actual availability and performance of applications from the end-users’ perspective. This helps you reduce the time and cost to identify and resolve problems.

Another option is to deploy real user monitoring (RUM) for internal applications. If using an application where you can insert code in the HTML headers, RUM will provide you relevant data and insight for users, whether they are remote or in a corporate location.

Last mile monitoring

Your end users aren’t connecting directly to a backbone provider. They connect to their local ISP or mobile provider via a “last mile” access network, which is why it is important to monitor from the last mile. Consumer ISPs do not consistently deliver the bandwidth they advertise as consumption peaks, and networks get saturated.

Performance degradation may not be a result of application or infrastructure issues but rather the end user’s network. Monitoring from the final leg of the network gives you visibility into performance issues outside of your network. While this may capture “noise” from the last mile network (and thus isn’t recommended alone for alerting), it provides valuable insights into the end-user experience. Don’t eliminate this perspective to reduce noise. Instead, correlate last mile data with monitoring from more consistent backbone and broadband provider locations to get a complete view of performance and improve your ability to verify and localize internet problems.

Mobile monitoring

More consumers are accessing websites and applications from their mobile devices. They expect the same level of performance when browsing on mobile as on their desktop or laptop.

Unfortunately, performance across a 3G or 4G/LTE network will not be the same as performance from a residential or business ISP. If your monitoring strategy does not include testing from mobile vantage points, you will have blind spots into the end user’s experience on these networks.

Catchpoint Node Types

Catchpoint offers a variety of node types to help you get a complete view of your end users’ experience. We recommend you use a combination of these to develop the most comprehensive monitoring strategy.

Backbone and Broadband Nodes

Get a clean view of performance by measuring digital services externally from datacenter-neutral facilities on tier 1 providers like Level3 or Cogent for Backbone, Comcast, or British Telecom for Broadband. Catchpoint users run most tests from backbone nodes because they provide the most consistent, trustworthy source of performance metrics. They are particularly useful for alerting with minimum false positives.

Enterprise Nodes

Location matters when it comes to internet connectivity and user experience, which is why we offer the ability to monitor applications from anywhere you want. With Enterprise nodes, you choose where and how to deploy. Run tests from inside the firewall, first mile data center, branch offices, or even corporate buses and airplanes. Detect problems your employees or customers experience on-site and localize whether a problem is internal or external.

Last Mile Nodes

Test from consumer ISPs for an accurate view of performance from the last mile. Last mile nodes are hosted in major geographies, across multiple last mile providers, on dedicated devices.

Mobile Nodes

Gain visibility into mobile end-user experience from synthetic tests run from multiple, worldwide cellular networks.

Cloud Nodes

If your primary goal is to understand and identify issues related to end-user experience, we encourage you to look at our backbone, last mile, and mobile nodes as they are in the path of service delivery. However, if you are looking at first mile testing or functional testing of cloud-based services from within the cloud, then the Catchpoint cloud nodes provide you with the data you need.

The chart below illustrates how TCP connection time can vary across these node types. For example, data from the last mile node shows the most delay and variability. This is to be expected as the last mile includes all the effects of a shared access network. Two cloud providers are also represented in the chart. Cloud (2) is the provider hosting the application. This shows a nearly flat connection time because it’s essentially just testing the internal connection within the cloud provider.

When choosing monitoring locations to understand the end user experience, you need to know if you are getting a complete picture or potential misinformation based on the vantage point of the node. If you are only monitoring from cloud providers where your application is hosted you won’t see the Internet-based delays or outages your end users are experiencing. For that, backbone nodes are generally the most consistent and reliable source.

The final perspective – Real User Monitoring

While not a node type, RUM is an integral part of a comprehensive monitoring strategy. Correlating metrics from real users with synthetic monitoring rounds out the performance view of your services, giving you a combined set of data from both passive and active sources.

Conclusion

Our comprehensive node types eliminate blind spots and deliver the insights you need to understand the end-users experience and to identify and resolve issues quickly. The ability to monitor from more than 700 backbone, broadband, last mile, wireless, cloud, and corporate locations can help companies detect suboptimal experiences from anywhere in real time. This holistic monitoring strategy is necessary to ensure the speed, reachability, reliability, and availability of all your digital services.

Ready to take Catchpoint for a test drive? Sign up for a free trial today.