I measured how many SaaS companies run A/B tests. My first answer was wrong by 84%, and the reason is the same one that ruins most tests.

I wanted a simple number: what share of large software companies actually run A/B tests.

You cannot ask them, so you look for the tool. Every client side testing platform loads a script, and scripts are visible to anyone who fetches the page.

Twenty minutes later I had my answer. Thirteen of twenty-eight sites, 46%. It felt about right, which should have been the first warning.

The number I nearly published

Here is the pattern list I started with. Read the third line before you read anything else.

Optimizely     optimizely|cdn.optimizely
VWO            visualwebsiteoptimizer|vwo.com|_vwo
Adobe Target   adobedtm|at.js|tt.omtrdc
Statsig        statsig

at.js is the historical filename of the Adobe Target library. It is also two letters, a dot, and two more letters. It matches any bundle that happens to contain that string, and modern JavaScript bundles contain a great many strings.

That one fragment attributed Adobe Target to four companies. I checked each one by hand before writing, because four felt like a lot for a tool that mostly lives in enterprise retail.

Three of the four had no Adobe anything on the page. The fourth loads Adobe Launch, which is a tag manager, not a testing tool. It may deliver Target. It may deliver analytics. From outside you cannot tell.

What strict matching gives instead

I rewrote every pattern to match only a vendor hostname, the one thing that cannot appear by accident.

Optimizely     cdn.optimizely.com | optimizely.com/js
VWO            dev.visualwebsiteoptimizer.com
Adobe          assets.adobedtm.com | .tt.omtrdc.net
Statsig        cdn.statsig.com | featuregates.org

The headline moved by 84% of its own value. Same pages, same day, same twenty-eight companies. Only the definition changed.

What is actually there

Under strict matching, seven sites carry a client side testing or tag tool:

Statsig, LaunchDarkly, Split, GrowthBook, PostHog, Convert, AB Tasty and Kameleoon appeared zero times. Some of those are real businesses with real customers. They are simply not on these twenty-eight pages, or they are used in a way that leaves no client side trace.

Why 25% is still not the answer

Two reasons, and they push in opposite directions.

It overstates. A script tag means the tool is installed. It does not mean an experiment is running. A company can carry Optimizely for two years while the experimentation programme quietly dies with the person who owned it. Presence is the weakest evidence of practice.

It understates, probably more. Server side testing, edge assignment and feature flags evaluated in a backend leave nothing in the HTML. A company running the most sophisticated experimentation programme in this sample would be invisible to my method, and the more mature the practice, the more likely it has moved server side.

So the honest claim is narrow: at least a quarter of these sites carry a client side testing tool. Anything stronger is not supported by what I did.

The part that generalises

The two numbers differ by nearly a factor of two, and I could have shipped either. Nothing in the process would have stopped me. The first number came from a working script that produced a plausible result.

That is the same failure that ruins experiments, arriving one step earlier. A test that measures the wrong event, or attributes to the wrong session, or counts a bot as a visitor, also produces a plausible result. It looks like a finding, it goes into a deck, and nobody checks the definition because the number felt about right.

The habit worth stealing is small. Before you believe your own measurement, take the three most surprising rows and verify them by hand. Not a sample. The surprising ones, because a false positive almost always shows up as something unexpected.

Four companies that had no business using Adobe Target was surprising. That is what saved the article.

What I would do differently

If I ran this again I would add a second pass that fetches the suspected vendor script and confirms it returns something from that vendor. A hostname can be present in a comment, a CSP header, or a preconnect hint without anything loading.

I would also stop treating one page as a site. I checked the homepage and the pricing page. Testing tools frequently load only on specific flows, and a checkout is where most of them actually live.

Neither change would make 25% correct. Both would make its error bars honest, which is the most any outside measurement of this can claim.

Questions people ask about this