Have you ever wondered exactly what automated synthetic transactions involve and why automated testing is important? If so, you’re not alone. Identifying problems with your website or software before end-users notice performance issues is crucial for long-term success. Discovering issues as they arise, however, isn’t easy — even with dedicated QA resources. This is especially true when your products have complicated functionality reliant on performant APIs and third-party dependencies. In this article, we’ll explore how you can address the issue of testing-at-scale using synthetic transactions.
Synthetic transactions are the result of scripts simulating activity normally performed on an application or website by real users. These transactions are mainly used to test the performance of production environments, but they are also useful for testing the functionality of updated applications prior to deployment.
For example, you can create a script that simulates a user’s connection to your website through a defined point of entry. This script could also include any subsequent actions or navigations as a follow-up. In this case, you could use the script to validate that all users have definite access to certain resources regardless of their location or entry to your services.You could also use it to validate user roles and permissions.
Another example is simulating database connections. Using synthetic transactions that run at certain intervals allows you to see how your website or database would perform under various loads representing real user activity.
The main similarity between synthetic transactions and API monitoring is that they both involve monitoring using software instead of real people. Also, they both verify if the application performs as expected and delivers a quality user experience.
Here’s how they differ:
Synthetic transactions monitor applications which may use APIs. API transactions monitor an API usable by many applications. Together, they provide a comprehensive view of performance across a transaction’s full workflow.
If you are just starting with synthetic transactions, here are a few important tips and best practices for you to follow when building out your synthetic transaction library.
Hunting for the right monitor, especially when you are in a hurry, can get pretty overwhelming. It’s a common problem, especially if you have multiple monitors that target the same service. For example, let’s say you have a ping check that tests availability every minute. Then, you have a monitor that tests the full user workflow. These two are obviously different tools, but you can hardly tell them apart if they target the same production services.
In order to avoid this issue, adopt a naming convention for transactions that’s based on clear and precise guidelines. For example, a convention that’s based on Type - Service - Env, would translate into Ping - Webstore - Production as your synthetic transaction’s name.
Setting up role-based access (i.e., permissions) to sensitive data is optional, but encouraged. Granting all internal users access to all transactions is not ideal, especially when some scripts can contain or fetch sensitive data. For example, some synthetic monitoring tools do not have the ability to obfuscate credentials, which may alter how you assign role-based access.
The more monitors and users your organization has, the more important it is to use permissions. Role-based access is typically based on user groups. Restrict groups using the least privilege principle to segment access to your synthetic transaction library. That will help you better protect sensitive data and provide a secure experience.
Many applications rely on external APIs to enhance their user experience. Weather information, music, meetings on a calendar — all of these bits of information require your services to communicate accurately with third-party applications. Submitting synthetic transactions to these external endpoints can help you monitor third-party applications and catch important changes, such as major or minor API version updates affecting request and response structures.
Synthetic transactions also enable you to monitor the performance of your vendors and watch for SLA breaches. Having unbiased historical data on record is essential to confidently holding your vendors accountable to their commitments.
Of course, synthetic monitoring is important at any stage of development. However, it’s exceptionally useful for testing new features before your website or application goes live. Synthetic monitoring is an ideal way to discover possible errors and obstacles that the users might encounter. For example, synthetic transactions can quickly validate that all input fields for a given feature are accepting the right input type’s list of all expected enumerations.
As edge computing extends reachability, it becomes increasingly difficult to test all relevant locations without using synthetic monitoring. For example, you may need to check potentially thousands of edge points of presence to validate that a critical update was received. That’s what makes automated monitoring edge-to-edge so critical.
Testing from multiple locations ensures end users have both access and acceptable interaction times with your applications. For example, say that you’ve added a new store location to your app that users can choose when browsing available goods or selecting pickup options. You can use synthetic transactions to test that the newly added store population in the “stores near me” results in Google and that all shopping options are available. At the same time, remember to test regions that represent not just the core of your user base, but also any emerging regions where usage is increasing.
Setting up synthetic transaction monitoring with a scope similar to your reach is the only way to fully validate continual first- or third-party updates.
Define thresholds for the performance and availability metrics of your synthetic transaction so your team can quickly identify problematic trends. Define both globalized (all locations violating) and localized (
You can get notified of the performance drop in numerous ways. For example, your alerting pathways can utilize SMS, email, push notifications on your mobile device, or even an application such as Slack, OpsGenie, or PagerDuty.
Segmenting alerts by certain groups can also be helpful if the ownership of your applications is divided by different engineering or product teams. For example, a CDN alert can be sent to a dedicated CDN team, while a problem with a social tag may be sent to a campaign manager.
Setting up role-based ownership and urgency labels helps employees understand what they need to pay attention to. Channels that are too noisy may get less attention. Dedicated alerts, however, feel more urgent.
Whenever possible, make sure that your synthetic transactions can run independently and in any order. In other words, avoid test dependencies in order to unblock your vision of how the whole application is performing when a given test fails. It also allows you to run many tests at the same time without having to wait for other tests to complete.
This section is dedicated to some of the challenges and pitfalls you might face when managing synthetic transaction testing.
Many websites use Captchas, which are designed to prevent the type of automated (bot) monitoring that is at the core of synthetic transaction monitoring. This makes relying on synthetic transactions for performance data more difficult. You can get around Captchas by either disabling them in your test environment or adding a hook to allow tests to bypass them.
Two-factor authentication (2FA) works by creating a temporary, one-time password using certain mobile apps, SMS, or email. It is difficult to factor into automated testing.
You can disable 2FA in a test environment, or disable it for certain users, or disable 2FA for users who log in from certain locations (via IP addresses). However, considering the importance of data security, it’s highly recommended that you select a synthetic monitoring tool that has the ability to incorporate 2FA as part of your testing selection criteria.
We don’t recommend logging into popular websites such as Facebook and Gmail using tools like Selenium’s WebDriver API. Not only is it slow, but you can also get your account banned.
It’s much better to use APIs that email providers offer, or the Developer Tools Service in the case of Facebook logins. It might seem like extra work to become familiar with the APIs instead of logging in via the UI, but in return, you will get more speed and reliability.
Now that we’ve gone over the do’s and dont’s of synthetic transactions, let’s review what features a good synthetic transaction monitoring tool should have.
As you can see, using synthetic transactions for your website or application is crucial for optimal user-experience. Synthetic transactions ensure not only the health of your application in a production environment, but also the validity of the external endpoints and data it requests across your features.
Moreover, synthetic transactions empower development and pre-production QA testing through an expandable library of transactions that test both basic functionality and performance under stressful loads. In other words, when you use synthetic transactions to test before you implement releases, your releases will be of increasingly higher quality over time.
Develop a killer DEM strategy.
Our one-page checklist will help you determine your monitoring strategy and data analysis essentials.
"With Catchpoint we can proactively monitor everything web—FTP and HTTP requests, APIs, or multi-step transactions. Their UI is very intuitive and flexible, and their analytics are very impressive."
Learn how Verizon DMS uses our Synthetic Monitoring solution
