Have you ever wondered exactly what automated synthetic transactions involve and why automated testing is important? If so, you’re not alone. Identifying problems with your website or software before end-users notice performance issues is crucial for long-term success. Discovering issues as they arise, however, isn’t easy — even with dedicated QA resources. This is especially true when your products have complicated functionality reliant on performant APIs and third-party dependencies. In this article, we’ll explore how you can address the issue of testing-at-scale using synthetic transactions.
{{banner-16="/design/banners"}}
What Are Synthetic Transactions?
Synthetic transactions are the result of scripts simulating activity normally performed on an application or website by real users. These transactions are mainly used to test the performance of production environments, but they are also useful for testing the functionality of updated applications prior to deployment.
For example, you can create a script that simulates a user’s connection to your website through a defined point of entry. This script could also include any subsequent actions or navigations as a follow-up. In this case, you could use the script to validate that all users have definite access to certain resources regardless of their location or entry to your services.You could also use it to validate user roles and permissions.
Another example is simulating database connections. Using synthetic transactions that run at certain intervals allows you to see how your website or database would perform under various loads representing real user activity.
{{banner-17="/design/banners"}}
Synthetic Transactions vs. API Monitoring
The main similarity between synthetic transactions and API monitoring is that they both involve monitoring using software instead of real people. Also, they both verify if the application performs as expected and delivers a quality user experience.
Here’s how they differ:
- Synthetic monitoring: Mimics application behavior through simulations and emulations.
- API monitoring: Validates the internal and third-party APIs used for communication.
Synthetic transactions monitor applications which may use APIs. API transactions monitor an API usable by many applications. Together, they provide a comprehensive view of performance across a transaction’s full workflow.
Best Practices for Synthetic Transactions
If you are just starting with synthetic transactions, here are a few important tips and best practices for you to follow when building out your synthetic transaction library.
Define a Naming Conversion for Transactions
Hunting for the right monitor, especially when you are in a hurry, can get pretty overwhelming. It’s a common problem, especially if you have multiple monitors that target the same service. For example, let’s say you have a ping check that tests availability every minute. Then, you have a monitor that tests the full user workflow. These two are obviously different tools, but you can hardly tell them apart if they target the same production services.
In order to avoid this issue, adopt a naming convention for transactions that’s based on clear and precise guidelines. For example, a convention that’s based on Type - Service - Env, would translate into Ping - Webstore - Production as your synthetic transaction’s name.
Assign Permissions to Protect Sensitive Data
Setting up role-based access (i.e., permissions) to sensitive data is optional, but encouraged. Granting all internal users access to all transactions is not ideal, especially when some scripts can contain or fetch sensitive data. For example, some synthetic monitoring tools do not have the ability to obfuscate credentials, which may alter how you assign role-based access.
The more monitors and users your organization has, the more important it is to use permissions. Role-based access is typically based on user groups. Restrict groups using the least privilege principle to segment access to your synthetic transaction library. That will help you better protect sensitive data and provide a secure experience.
Monitor Third-Party Applications
Many applications rely on external APIs to enhance their user experience. Weather information, music, meetings on a calendar — all of these bits of information require your services to communicate accurately with third-party applications. Submitting synthetic transactions to these external endpoints can help you monitor third-party applications and catch important changes, such as major or minor API version updates affecting request and response structures.
Synthetic transactions also enable you to monitor the performance of your vendors and watch for SLA breaches. Having unbiased historical data on record is essential to confidently holding your vendors accountable to their commitments.
Test New Features
Of course, synthetic monitoring is important at any stage of development. However, it’s exceptionally useful for testing new features before your website or application goes live. Synthetic monitoring is an ideal way to discover possible errors and obstacles that the users might encounter. For example, synthetic transactions can quickly validate that all input fields for a given feature are accepting the right input type’s list of all expected enumerations.
Test From Edge to Edge
As edge computing extends reachability, it becomes increasingly difficult to test all relevant locations without using synthetic monitoring. For example, you may need to check potentially thousands of edge points of presence to validate that a critical update was received. That’s what makes automated monitoring edge-to-edge so critical.
Testing from multiple locations ensures end users have both access and acceptable interaction times with your applications. For example, say that you’ve added a new store location to your app that users can choose when browsing available goods or selecting pickup options. You can use synthetic transactions to test that the newly added store population in the “stores near me” results in Google and that all shopping options are available. At the same time, remember to test regions that represent not just the core of your user base, but also any emerging regions where usage is increasing.
Setting up synthetic transaction monitoring with a scope similar to your reach is the only way to fully validate continual first- or third-party updates.
{{banner-3="/design/banners"}}
Set Alerts
Define Performance Thresholds
Define thresholds for the performance and availability metrics of your synthetic transaction so your team can quickly identify problematic trends. Define both globalized (all locations violating) and localized (
Choose Alerting Pathways
You can get notified of the performance drop in numerous ways. For example, your alerting pathways can utilize SMS, email, push notifications on your mobile device, or even an application such as Slack, OpsGenie, or PagerDuty.
Setup Role-Based Ownership/Urgency Labels
Segmenting alerts by certain groups can also be helpful if the ownership of your applications is divided by different engineering or product teams. For example, a CDN alert can be sent to a dedicated CDN team, while a problem with a social tag may be sent to a campaign manager.
Setting up role-based ownership and urgency labels helps employees understand what they need to pay attention to. Channels that are too noisy may get less attention. Dedicated alerts, however, feel more urgent.
Avoid Test Dependency
Whenever possible, make sure that your synthetic transactions can run independently and in any order. In other words, avoid test dependencies in order to unblock your vision of how the whole application is performing when a given test fails. It also allows you to run many tests at the same time without having to wait for other tests to complete.
{{banner-18="/design/banners"}}
Pitfalls for Synthetic Transaction Testing
This section is dedicated to some of the challenges and pitfalls you might face when managing synthetic transaction testing.
Captchas
Many websites use Captchas, which are designed to prevent the type of automated (bot) monitoring that is at the core of synthetic transaction monitoring. This makes relying on synthetic transactions for performance data more difficult. You can get around Captchas by either disabling them in your test environment or adding a hook to allow tests to bypass them.
Two-Factor Authentication
Two-factor authentication (2FA) works by creating a temporary, one-time password using certain mobile apps, SMS, or email. It is difficult to factor into automated testing.
You can disable 2FA in a test environment, or disable it for certain users, or disable 2FA for users who log in from certain locations (via IP addresses). However, considering the importance of data security, it’s highly recommended that you select a synthetic monitoring tool that has the ability to incorporate 2FA as part of your testing selection criteria.
Social Logins
We don’t recommend logging into popular websites such as Facebook and Gmail using tools like Selenium’s WebDriver API. Not only is it slow, but you can also get your account banned.
It’s much better to use APIs that email providers offer, or the Developer Tools Service in the case of Facebook logins. It might seem like extra work to become familiar with the APIs instead of logging in via the UI, but in return, you will get more speed and reliability.
{{banner-19="/design/banners"}}
Key Functionalities of Synthetic Transaction (Monitoring) Tools
Now that we’ve gone over the do’s and dont’s of synthetic transactions, let’s review what features a good synthetic transaction monitoring tool should have.
- Support for multiple types of logins: Enables you to login using SSO, Citrix, XenDesktop, AWS Workspaces, etc.
- Simulation of different protocols: Enables you to leverage REST, JSON, YAML, HTTPS (web), SMTP (email), ICA (Citrix), SQL (database) for various situations.
- Record and replay: Enables you to record a user-driven transaction and replay it instead of writing a script. Revision to certain classes, without re-recording the whole transaction, should also be possible.
- Smart element locator: Enables you to find UI elements as the design changes over time.
- Conditions: Enables you to direct transactions to wait for the completion of another step or even loop the transaction a certain number of times.
- Support for various types of variables: Enables you to reuse tests dynamically based on data like the location.
- Reusable steps: Enables you to reuse and reassemble simulation components to build new tests.
- Continuous integration: Enables you to run scripts triggered by events in your DevOps continuous delivery processes.
- Assertion and Validation: Enables you to validate a boolean expression to be true or false before proceeding. The assertion validation can be a 200 HTTP response, a JSON payload, or in a form of an image or text on the website. You also need a Safe Harbour feature to make sure that content complies with privacy laws in a specific country. In addition, assertions can be used in the absence of certain elements, such as an out-of-stock item on e-commerce websites and stores.
- Step-by-step measurement and completion events: Enables you to record a separate measurement for each step. The completion event, for example, lets the synthetic monitoring node know when the monitoring is complete.
- Step-by-step analytics: Enables you to evaluate the performance and availability data for both an entire transaction and its individual steps.
- Payload execution time extraction: Enables you to execute and extract custom payloads, and separate them from your total transaction times.
- Support for Selenium: Enables you to quickly compose tests referencing a list of functions and examples that populate the IDE.
- Data grouping: Enables you to compartmentalize data subsets into themed measurement units (e.g., tied to a CDN or to infrastructure)
{{banner-sre="/design/banners"}}
Conclusion: Using Synthetic Transactions is Critical
As you can see, using synthetic transactions for your website or application is crucial for optimal user-experience. Synthetic transactions ensure not only the health of your application in a production environment, but also the validity of the external endpoints and data it requests across your features.
Moreover, synthetic transactions empower development and pre-production QA testing through an expandable library of transactions that test both basic functionality and performance under stressful loads. In other words, when you use synthetic transactions to test before you implement releases, your releases will be of increasingly higher quality over time.