<div anchor>Introduction</div>
The most expensive experiments in publishing aren't the ones that fail, they're the ones that never run.
In 2012, a Microsoft employee working on Bing had an idea about changing how the search engine displayed ad headlines. It was a small change, maybe a few days of engineering time. But it was one of hundreds of ideas in the backlog, and the program managers deemed it low priority. So it sat there for more than six months.
Eventually, an engineer who noticed the implementation cost was trivial decided to run a quick A/B test. Within hours, the new headline variation was producing abnormally high revenue, triggering an internal alert that usually signals a bug, but it wasn't one. The change increased revenue by 12%, worth over $100 million annually in the US alone.
Stefan Thomke, the Harvard Business School professor who documented this story, draws a specific lesson from it. The problem wasn't that Microsoft lacked good ideas. The problem was the gap between having an idea and being able to test it. The idea existed for six months before anyone ran the experiment.
Thomke calls the broader principle "high-velocity incrementalism": most progress comes from implementing hundreds or thousands of minor improvements, but only if you can test them fast enough for the results to compound. Booking.com runs roughly 25,000 experiments a year, with around 1,000 running concurrently at any given time. About 90% of those experiments fail. But the 10% that work compound into enormous performance gains, because the infrastructure lets the next experiment start before the last one has finished being analysed. How many monetization experiments did your organisation run last year?
<div anchor>The Six-Week Idea</div>
The six-week idea
The first post in this series argued that the paywall's black and white view of the world no longer covers the range of visitors arriving at publisher and media sites. The second made the case that entitlements, the granular permissions governing who gets what under which conditions, are the atomic unit of every access decision, and that infrastructure flexibility determines commercial flexibility.
This post is about what happens in the gap between having a commercial idea and being able to execute it. Because in our experience, that gap is where most publisher revenue never grows to reach its full potential.
The pattern is consistent enough across the publishers we talk to that it's worth describing concretely. A commercial team identifies an opportunity: maybe a trial offer for readers who've hit the paywall three times in a week, or a regional price for a market where the standard rate doesn't match local purchasing power, or metered access terms for an AI agent under a licensing agreement (the kind of visitor we'll dig into in part four). It's a good idea. It gets written up, builds internal support, and arrives at the engineering team.
That's where the timeline changes. Not because engineers are slow. In our experience, publisher engineering teams are talented, and stretched. The problem is that most monetisation platforms couple access logic tightly to application code, which means changing a price travels through the same deployment pipeline as shipping a product feature: backlog, sprint, review, staging, QA, release. It’s not a configuration change, it is a release.
<div anchor>What Booking.com Knows</div>
What Booking.com knows that publishing doesn't (yet)
Thomke's research at Booking.com revealed something counterintuitive about experimentation culture. The companies that test the most aren't the ones with the best ideas. They're the ones whose infrastructure makes testing cheap and fast enough that the quality of any individual idea matters less than the volume and speed of the cycle.
At Booking.com, no change ships without an A/B test, including changes proposed by the CEO. (Their first American CEO suggested a logo change that never made it to production because the test didn't validate it.) The infrastructure is designed so that product managers can run experiments safely without breaking the site, with guardrails, holdout groups, and real-time metrics monitoring built into the platform.
The publishing industry has a version of this problem, but the constraint is more specific. Publisher engineering teams aren't blocking commercial innovation on purpose. They're routing every commercial decision through a technical pipeline because the architecture offers no other alternative. The platform and infrastructure doesn't allow them to distinguish between "change a price" and "ship a feature."
The compounding cost matters more than the delay on any individual project. Nobody can track and document the revenue from experiments that never ran and innovation that never happened.
<div anchor>The Counterargument</div>
The counterargument (and why it's half right)
The strongest pushback on this is that speed without governance and guardrails is dangerous. Any CTO who's been asked to hand commercial teams the key to production systems will rightly push back. Pricing logic that's wrong or broken can destroy subscriber trust overnight, or create untold reputational damage. Letting in a hostile user agent means your entire back catalogue might get stolen. Entitlement rules that conflict can create access holes. Experimentation without statistical rigour produces noise, not signal.
Booking.com doesn't achieve 25,000 experiments a year by removing engineering guardrails. They achieve it by building infrastructure that separates what's configurable from what's structural. Engineers define the feature set, the properties those features can have, and the rules for how properties interact. They set the boundaries. Commercial teams then operate within those boundaries, configuring plans, pricing, offers, and access rules through an interface designed for how they work, not through the deployment pipeline.
The result is that both teams get what they actually need. Engineers retain control over the technical foundation: programmable, version-controlled, testable infrastructure. Commercial teams get the autonomy to act on their ideas at the pace those ideas require. Changes go live in minutes. If something doesn't perform, it gets rolled back instantly. No tickets filed. No sprints waited on.
This separation is what makes the difference between a system that evolves with the business and one that constrains it. And it's the same architectural principle that governs at Spotify, YouTube or Netflix: the commercial logic and the engineering infrastructure operate independently within a shared framework of trust.
Booking.com is a particularly relevant comparison for publishers because they occupy a structurally similar position. They're an aggregator, but they're also being aggressively aggregated by Google. They face the same tension between owning the customer relationship and having their value extracted by a larger platform. Their Genius loyalty programme, where members now account for more than 50% of booking volume, shows that membership and loyalty can be built even in a highly transactional category. That programme started as an experiment. Agents and bots will transform their world too, but because the experimentation infrastructure already exists, they'll be adapting while others are still filing tickets.
<div anchor>The Question For Your Organisation</div>
The question for your organisation
Thomke's research found that 77% of economic growth comes from improvements to existing products, not from breakthrough innovations. Lego, after its near-bankruptcy in 2004, implemented a system of structured incremental improvements that drove 95% of annual sales and restored profitability. The pattern holds across industries: the organisations that capture the most value aren't necessarily the ones with the best single idea. They're the ones that can continually empower their teams to test, learn, and iterate faster.
For publishers, this raises a specific question. If the Bing headline change sat in a backlog for six months and turned out to be worth $100 million a year, what's sitting in your backlog right now that could transform your revenue? And what would it take to find out whether it works or not?
The next post in this series shifts from human audiences to the other half of the traffic equation. Bots, crawlers, and AI agents now account for over half of all web traffic, and the publishers who treat machine visitors as a customer category rather than a security threat are the ones building the next meaningful revenue line.
About MonetizationOS
MonetizationOS is an intelligent, edge-native infrastructure layer that governs and monetises every access request, human and machine, in real time. Get started for free at monetizationos.com.
