What is Synthetic Transaction Monitoring?
I used to work at a large multinational.
We’d deploy new versions of our digital assets all the time. Before doing that, we’d of course make sure all unit tests passed. A large “go/no go” meeting took place where we’d test everything (or so we thought). We even hired third-party vendors to QA our products. We’d sit in a “war room” with multiple dashboards showing database latencies, error logs, and the like. We’d tune our “early warning” alerts to send SMS and emails at the slightest error.
But guess what?
Whenever we’d launch, call center volume was the true “acid test” as to whether the launch was successful. Alas, about 20% of the time a spike in calls occurred indicating a problem. And even if it wasn’t immediate, a week or two later, and despite having tuned DevOps, the volume of calls increased as portions of the site failed.
Our customers were our QA team.
This made me feel like a failure. But, unless you’re one of the top tech companies, this issue is quite common. (And even those top tech companies often fall flat.)
Today’s web apps and sites are complicated: multiple microservices, databases, front-end frameworks, API gateways, CDNs, and beyond. While each of these is often well-monitored, failure points expand with more systems.
Synthetic Transactions Monitor the Integrity of Your Customer-Facing Digital Presence
“Synthetic transaction monitoring.” It’s a rather artless name, isn’t it? (It’s sometimes also called “web transaction monitoring.”) Yet it’s a core part of what Fluxguard provides. And we built our product to ameliorate the problems I described above. So, what does synthetic transaction monitoring mean? And do you need it?
Synthetic transaction monitoring = end-user experience monitoring.
In essence, synthetic transaction monitoring automates a web browser and pretends to be a user on a web app or site. It explores key areas. Logs in. Visits dashboards. And it reports problems to the site owner. This way, the end-user experience is better guaranteed to function.
Indeed, synthetic transaction monitoring means automated oversight of key areas of a web app or site. “Synthetic,” after all, implies “not real”: this rather clunky word means monitoring of transactions not initiated by actual users. Good! It’s best for these “fake users” to discover problems versus genuine people.
“Transaction,” in this context, denotes an important activity on a web app or site. For instance, it could signify logging in to a site, placing an order, or checking a bank balance. It can also represent visiting a site and exploring key pages.
Synthetic Monitoring Goes Beyond Simple Uptime Monitors
Modern web apps and sites rely on a host of tests and monitors to better ensure that end-users have an optimal experience.
Some of these were described in the intro, and a typical test “stack” might include:
- Unit tests
- Integration tests
- End-to-end tests
- CI/CD-triggered deployment tests
These inspections are a vital part of test-driven development (TDD) that is today’s industry standard. For most of these tests, developers will write “assertions”: for example, “this function should return a string.”
Since these examinations are run during development or at deployment, they often cannot reveal “in the wild” bugs or errors. Moreover, since these tests are assertion-based, in many cases they will not identify “unknown” bugs. That is to say, they require the developer to think about results.
Let’s now turn our attention to uptime “stacks.” These include:
- Uptime monitors
- Application performance monitoring (APM)
- API endpoint monitoring
- Log file alerts
- Database and network activity monitoring
And so on. Many times, he modern web app or site has several “monitors” to help guarantee end-user experiences.
However, most of these monitors don’t pledge end-user satisfaction. Why? These monitors are oblivious to the end-user. Sure, uptime monitors will announce when a page returns a 404 Not Found or 503 Internal Error message. Or a log file alert might reveal that you are out of disk space. Or you might get a warning that your database I/O is saturated.
All of these issues are important, true. But modern web apps and sites need much more. Further, in most cases these monitors are inspecting an individual page rather than a sequence of pages. Indeed, today’s apps often rely on multi-page, multi-step transactions; simple uptime monitors are not useful.
Consider all of the ways an end-user can have a sub-optimal experience on your web app or site:
- Images aren’t loading from a CDN
- API is up, but returning incorrect data
- Third-party Javascript files are unavailable/corrupted/changed in a way that affects your site
- Front-end framework updates have adverse effects
- A multi-step transaction breaks, but only for some users
- CSS changes have unexpected effects on other areas of your site
- Old Javascript code is removed... but, wait! It’s still needed
- A new logo is a different dimension and it is cut off in the top nav on some pages
- New email form validation breaks for users with non-.com email addresses
- Cookies aren’t expiring on logout
Today’s complex websites can fail for many reasons. And few of the “traditional” means of monitoring (uptime, database, log file, API) can identify them.
“Fake” Users Detect Genuine Bugs
As mentioned above: actual users should not do QA. Instead, we can rely on “synthetic” users!
Let’s take a look at how we use Fluxguard to monitor our front-end user experience.
In the screenshot above, you can confirm that we monitor a series of pages. Cookies, “local storage”, and “session storage” are preserved throughout. This allows us to construct and monitor multi-step and multi-page transactions, such as customizing and ordering a product.
This ability to oversee a complex, multi-step transaction is essential. Instead of scrutinizing one or two pages, we’re instead monitoring a sequence of pages. This, of course, better mimics how users interact with our digital assets: they don’t visit just one page and leave. Rather, they go to the home page, log in, click a few places, and so on. As such, synthetic transaction monitoring better meets the requirements of today’s modern web.
In this screenshot, when we click on any of the pages from our monitored sequence, we can add multiple actions to perform on that page. This gives us great flexibility in examining all sorts of transactions: logins, form submissions, clicking through a carousel, and so on.
Testing for “Unknown” Bugs
So what’s the end-product of synthetic transaction monitoring? It depends on the solution. Ultimately, it will come down to one of two solutions: assertion-based tests or exploratory testing. Let’s consider each in brief.
Assertion-based synthetic transaction monitoring requires more set-up, as the customer has to set specific “assertions” that must pass each page. For instance, after submitting a shopping cart, there should be a final total, a link to an invoice, and so on. Another hitch with this type of oversight is that it requires the customer to speculate what can go wrong with each page. The link to the invoice may be present. But perhaps the link to return the product isn’t?
We prefer exploratory synthetic transaction monitoring. In this case, the solution reports what has changed on a pixel, DOM, or text level. This way, you can identify all sorts of problems early: from CSS side-effects, to failing APIs, to malicious edits.
Avoid False-Positives When Monitoring Websites for Changes
A problem with exploratory testing and web change monitoring are false-positives. That is, getting alerts for minor issues, such as a side-bar with a new blog link. Fluxguard offers various innovative solutions to minimize false-positives:
- Exclusion Filters instruct Fluxguard to remove an entire area of a site (or page) before monitoring. This way, you can strip sidebars, footers, headers, or other areas of a page that change often, but which aren’t of critical importance.
- Inclusion Filters do the opposite of Exclusion Filters: they command Fluxguard to monitor a particular area for changes. This way, you can instruct Fluxguard to focus on key areas of a page – such as a dashboard – and disregard everything else.
- Transforms are a robust monitoring optimization within Fluxguard: instead of stripping areas of a page, Transforms will replace often-changing content types with a static string. For example, you can replace all numbers with the word “NUMBER.” This way, when monitoring a dashboard, for instance, you can still ensure that key areas display numbers as they should... but you won’t get an alert when numbers change.
- Network blocks command Fluxguard not to load certain files. It’s similar to Adblock. You can instruct Fluxguard not to load Google Analytics, for example. It minimizes DOM disruption and pollution from complex third-party Javascript snippets. (At the same time, you may prefer to not block third-party scripts so Fluxguard can audit what they’re doing!)
Putting It All Together: Synthetic Transaction Monitoring Is a Must For Key Digital Assets
It’s no longer sufficient to perform “sanity checks” after every deployment. Unless your web app is uncomplicated, it’s not possible to replicate all the possible circumstances your customers might find themselves in.
You can no longer rely on simple “uptime monitors” to check your site for errors. Your site may be “up,” but your API might be down. Or your site and API may be up, but your API may be returning faulty data. Or a third-party Javascript file may be inadvertently breaking the DOM.
Non-stop exploratory testing of your production site is essential. And we believe that open-ended change comparison tests – comparing code, content, and design from visit to visit – are the necessary way to conduct these assessments.
Synthetic transaction monitoring is a must for key digital assets. This ensures that your web app is performing as expected, quickly and accurately responding to customer requests without fail. This type of monitoring tests your application’s workflow from beginning to end. If a problem occurs at any point along the way, you can be alerted in real-time, allowing you to take corrective action right away. By using synthetic transaction monitoring, you can gain valuable insights into the performance of your web app and protect your customers from potential issues.