Will AI Replace Growth Marketers?

Thirty-one rows in the experiments tab. You wrote nineteen of them in a single afternoon in March, and it felt like the most productive afternoon of the quarter.

Four have shipped since.

The rest sit there, each one perfectly reasonable, waiting for a slot that keeps not arriving. Meanwhile someone in your feed is explaining that they generated four hundred ad variants overnight, and you’re doing quiet arithmetic about what that would even mean for a page that gets nine hundred visitors a week.

That gap between the ideas and the slots is the actual story of AI in growth marketing, and almost nobody is telling it, because the interesting version involves sample sizes.

Most good ideas fail, which is why the shots matter more than the ideas

Start with the number that makes growth marketing a strange profession. When Ron Kohavi and Stefan Thomke wrote up Microsoft’s experimentation program for Harvard Business Review, the finding was that roughly a third of well-designed ideas improved the metric they were built to improve. A third did nothing measurable. A third made things worse.

These were not lazy ideas. They were the ones a serious team believed in enough to build.

If two out of three good ideas fail, then the value of a growth program was never in having ideas. It was in how many you could afford to find out about, and that number has always been set by traffic, because statistical power is paid in visitors.

Now put AI next to it. Producing a variant went from two days to two minutes, and a whole lifecycle sequence went from a month to an afternoon.

The number of people arriving at your funnel each week did not move by one.

So the ratio inverted. The bottleneck used to sit in production, which made speed the valuable skill. It now sits entirely in selection, and the valuable skill is deciding which four of thirty-one rows deserve the only traffic you have.

None of this is the occupation shrinking. The Bureau of Labor Statistics projects market research analysts and marketing specialists, the category most growth marketers are counted in, to grow about 7% from 2024 to 2034, a rate it classes as much faster than average, on roughly 87,200 openings a year.

Talk to enough growth teams and the same shape turns up. The experiment backlog got two or three times longer over the past year and the count of tests that actually shipped barely moved, and nobody has quite named why that feels bad.

If your backlog grew faster than your traffic did this year, that’s worth putting a number on rather than carrying around, and the 3-minute readiness check works out how much of your week goes into producing tests against how much goes into choosing them.

What AI cannot do in a growth program

Three things, and generating copy is conspicuously not among them.

The first is send anyone to your site. That sounds too obvious to write down until you notice how much AI commentary assumes faster production means faster learning, when the thing you’re waiting on is a sample filling up. A test on a page converting at 3% that needs to detect a half-point improvement wants roughly eighteen thousand visitors per arm, and at twelve hundred visitors a week that’s half a year for one answer. No model shortens it by a day.

The second is decide which idea gets the slot. Ranking thirty hypotheses looks like a sorting problem and isn’t, because the inputs sit outside any dataset. Which change engineering will actually build this quarter, which page your chief executive has feelings about, which idea already failed in 2024 under a different name.

The third is call a flat result flat. When a variant comes back a point ahead with a confidence interval straddling zero, a model will hand you a fluent paragraph describing an improvement, because that’s what the numbers say if you don’t understand what they are. Saying out loud, in a room that has been waiting three weeks, that the test answered nothing is not analysis. It’s someone putting their own credibility behind a number that disappoints people, which is not a thing you can ask software to do.

The Workplace AI Institute works with a lot of growth teams, and the ones who seem calmest about all this are the ones who can say out loud, without looking it up, roughly how many tests a year their traffic can answer. It’s a small number and knowing it changes everything downstream.

The arithmetic is the new craft, and it fits in one prompt

Here’s the ten-minute version. For each backlog item write down the page it affects, that page’s current conversion rate, the improved rate you’d want to detect, and roughly how many visitors it gets weekly. Then paste it in.

For each proposed test below, calculate the approximate visitors needed per variant using the standard rule of thumb for a two-arm test at 95% confidence and 80% power, which is 16 times p times (1 minus p) divided by the absolute minimum detectable effect squared, where p is the baseline conversion rate. Divide by the weekly traffic to get weeks required. Show your arithmetic. Give me a table with test name, visitors needed per variant, weeks required, and a verdict of runnable, slow, or unrunnable, treating over 8 weeks as slow and over 16 as unrunnable. Do not tell me whether any of these are good ideas.

What comes back is not a plan. It’s a list of the things in your backlog that were never going to produce an answer, which for most growth programs is somewhere between a third and half of it.

Three things to change before the next planning cycle

Size everything before anyone designs anything. Unrunnable tests don’t launch because people can’t do the math. They launch because the math happens after the design work, when three colleagues have already spent a week and nobody wants to say it needs seven months. Run it in the meeting where the idea comes up, on the back of an envelope.

Put a because clause on every hypothesis. Not “test a shorter form” but “if we cut the form to three fields, then paid signup rate rises, because 31 of 40 sales calls last quarter mentioned the length before anything else.” A flat result on the first teaches you nothing. A flat result on the second kills a belief, and killing beliefs is what compounds.

Make flat results reportable. If your program has never written down “inconclusive” as a verdict, flat results are being quietly relabeled as small wins, and every plan built on top of them inherits the error.

If you want the sizing method, the hypothesis form, and the readout structure worked through end to end with the prompts already written, the AI for Growth Marketers course covers all three across four units, along with the lifecycle and forecasting work that sits either side of them.

So will AI replace growth marketers?

No, and the occupation is growing while the tools get better, which is not a coincidence. Something has to decide where a company’s limited attention goes, and that something has to be answerable for the decision.

What is ending is the version of the job where being fast at producing things was the differentiator. That was always a proxy, and it lasted because it was the easiest thing to see from the outside.

In our experience the growth marketers who get promoted seldom have the best win rate, because a win rate mostly reflects how much traffic somebody was handed. They’re the ones whose flat results were still worth reading a year later.

If those thirty-one rows are what you keep scrolling past, the readiness check sorts what you actually did last month into the half a model can finish for you and the half that decides whether any of it counted.

The thirty-one rows aren’t a backlog. They’re a list of questions your traffic can only afford to answer four of this year, and getting to choose which four is a better job than the one where you were mostly busy.