Beach scene at Scheveningen, painting by J.H. Weissenbruch (1887)

AI SEO cleanup

AI made SEO cheap to ship. I clean up what it shipped.

A content pipeline or an agent can publish forty thousand pages, rewrite every title and edit robots.txt in one afternoon. I find out what it did to your website, put a number on the damage and remove it, without taking the pages that earn money along.

Websites with 50M+ pages Developer since 1997 Ranked in 244 countries

The pattern

It works for a few months. Then it stops, and often takes more than it gave.

Lily Ray followed more than 220 websites that AI content platforms had presented as customer stories. The number of pages grew for six to twelve months. Traffic peaked three to six months after that. Then most of it went.

sample

220+

Websites that scaled content with AI, taken from the platforms' own case studies.

lost 30% or more

54%

Of them lost at least 30 percent of their peak organic traffic.

lost half or more

39%

Lost at least half. Many ended below where they started.

lost 75% or more

22%

Lost three quarters or more of the peak.

Source: Lily Ray, “It Works Until It Doesn't”, May 2026. Traffic estimates from Ahrefs, checked against Sistrix. It is a correlation, and she says so herself.

Programmatic SEO is as easy as it has ever been. Once Google detects the pattern the whole set goes, and most of it never converted anyway.

Why it happens

A model gives you the average. Average SEO does not win.

A language model learns from everything that was ever written about SEO and answers with what is most likely. That is the average of the industry: every forum post, every outdated guide and every checklist copied from another checklist, weighted by how often it was repeated.

The average SEO makes a lot of mistakes. And search does not reward average work to begin with. There are ten places on a results page, and every competitor can ask the same model the same question.

  • Often repeated is not the same as correct. The most repeated advice wins inside the model, also when it is outdated.
  • Your competitors get the same answer. Whatever the model suggests to you, they publish as well.
  • It knows nothing about your website. No logs, no revenue, no record of what was tried before.

The risks

Four ways it goes wrong. Usually on the same website.

None of this is an argument against automation. It is what happens when nobody reviews what the automation ships.

01

Too many pages, competing for the same search

A generated page costs nothing, so every keyword variation gets one. Fourteen pages then answer the same question. Google shows a different one every week and ranks none of them well. The page that used to rank is in the same pile.

  • Cannibalisation. Several URLs trade places on one query, each with a fraction of the clicks.
  • Index bloat. Thousands of URLs that Google crawls and then declines to index.
  • Crawl waste. On a large website the new pages get fetched, and the ones that sell get fetched less often.

02

Hallucinations, published under your name

A language model writes what sounds right. In a draft that is an inconvenience. On a live website it is a statistic nobody measured, a price that was never in your feed, a feature the product does not have, or a link to a page that does not exist.

Ahrefs looked at 16 million URLs cited by AI assistants. Visitors ended up on a 404 page 2.87 times as often as they did from Google Search. The same models write your internal links.

  • Invented numbers, studies and quotes, with a confident source that does not exist.
  • Prices, specifications and policies that differ from the system that holds the real ones.
  • Structured data for ratings nobody gave.

Source: Ahrefs, “How Often Do AI Assistants Hallucinate Links?”, September 2025.

03

Accidents with the files that control everything

Agents now have write access to the CMS, the repository and sometimes Search Console. Most of what they change is harmless. A few files are different. One line in robots.txt, one noindex in a template or one uploaded disavow file changes the whole website, and nothing looks wrong on the page itself.

  • robots.txt. A staging rule, or a well-meant "hide the thin pages", reaches production.
  • noindex and canonicals. Set in a template, so set on every URL built from it.
  • Disavow files. Google calls it an advanced feature that can harm your performance when it is used incorrectly. Agents upload one to silence a "toxic links" warning in a tool.
  • Both at once. Google cannot read a noindex on a page it is not allowed to fetch.

Source: Google Search Console Help, “Disavow links to your site”.

04

Programmatic pages that get removed, and never converted

A template, a keyword list and a model, and the section exists by the end of the week. Detection got easier too. I see more and more cases where Google removes the whole set once it has recognised the pattern, the useful pages included.

National Today had more than 850,000 URLs indexed in one directory. After a manual action for scaled content abuse the section left the index, and its citations in ChatGPT went with it.

Then look at what those pages did while they still ranked. Usually very little: low-value visits from people who were never going to buy.

Source: Glenn Gabe, “When Mt. AI crumbles, ChatGPT can follow”, April 2026.

What else can go wrong

Nine more. Slower to show, and harder to trace back.

The first four are easy to recognise once you know them. These take longer to surface. Most have a public case or a written policy behind them.

−%

The whole website pays

A weak section does not fail on its own. In Lily Ray's sample the decline often ended below the level from before the content went live. Pages that had worked for years went down with the new ones.

Lily Ray, May 2026
[!]

A manual action

Google's policy on scaled content abuse covers pages made in bulk for rankings, "no matter how it's created". Generative AI is its first example. The consequence: ranking lower, or not appearing in results at all.

Google spam policies
llm:

AI visibility goes with it

The assistants lean on search results. When National Today lost its section in Google, Glenn Gabe found it gone from ChatGPT's answers for almost every prompt he tested. Content made "for AI search" is not exempt.

Glenn Gabe, April 2026
#1

GEO tactics that backfire

A list of "the best tools" with yourself at number one. A page per question. A comparison page per competitor. In Ray's data, sites with self-promotional listicles lost 40 to 95 percent between January and April 2026.

Lily Ray, May 2026
xx-XX

Translation without a reader

One model, thirty languages, and nobody on the team who speaks them. Ray lists AI translation without human review among the causes of sitewide quality problems. The language targeting across all those URL sets is usually unchecked too.

How I audit markets
{ }

Markup that claims too much

Generated JSON-LD is often invalid, or says things the page does not: ratings without reviews, authors who do not exist, answers that are not on screen. Rich results disappear, and misleading markup can get a manual action of its own.

My list of 17 mistakes
§

Legal exposure

In Moffatt v. Air Canada a tribunal held the airline to what its chatbot told a customer: a company answers for the information on its own website. Article 50 of the EU AI Act adds disclosure duties for some AI-generated text. I flag the pages. Your legal team decides.

Article 50, explained
diff

Rewrites of pages that worked

"Refresh all content" sounds harmless. The agent rewrites titles and copy on pages that ranked for years, and moves every date forward without a real change. Without a record of the old version there is nothing to roll back to.

js()

Vibe-coded rebuilds

The marketing website is rebuilt with an AI coding tool in a weekend. Content that only exists after JavaScript runs, routes behind a hash, no canonicals, and no redirects from the old URLs.

17 SEO mistakes LLMs make

The cleanup

How a cleanup runs. Stop the damage first, then decide per page type.

The order matters. Deleting pages before you know what the automation changed removes the evidence along with the problem.

  1. Freeze. The pipeline stops publishing. Agents lose write access to robots.txt, templates, redirects, Search Console and the disavow file. Nothing new goes live until we know what is there.
  2. Inventory. Every URL and every change the automation made, taken from the repository, CMS revisions, sitemaps, server logs and Search Console. If no change log exists, I rebuild one from those.
  3. Undo the accidents. Blocked sections, stray noindex and canonical tags, the disavow file. These are the cheapest repairs, and they recover first.
  4. Triage per template. Clicks, conversions, links and cannibalisation per page type. Each type gets a decision: keep, merge, rewrite or remove.
  5. Fact check what stays. Claims, prices, links and markup, checked against the systems that hold the truth: the product feed, the policy pages, the review platform.
  6. Remove in batches. Gone, redirected or noindexed, chosen per case. In batches and with the logs open, so a mistake costs one batch.
  7. Reconsideration. Only when there is a manual action. Google wants to see the problem gone and documented, so that is the request I write.
  8. Guardrails. Rules for what automation may change on its own, what needs review and what it never touches. Then checks that enforce them.

Triage per template, never per page

Nobody can review forty thousand pages one by one, and nobody has to. Generated pages come from a handful of templates, and each template has a record: what it earned, what it cost the pages around it, and whether anyone links to it.

  • Keep what earns its place. Some generated pages answer a real search and convert.
  • Merge before you delete. Forty near-identical pages can become one good one.
  • Remove with the right status code, and keep the redirects for pages that have links.
How I work on large websites

Afterwards

The automation stays. It gets rules.

I am not here to take the tools away. I use them every day. Drafting, clustering, checking ten thousand titles against a rule: all fine. The question is which changes may go live without a person looking. For a handful of files the answer is none.

  • A short list of files and settings that no agent writes to.
  • Review before publishing, by a named owner who can say no.
  • Checks in the release on robots.txt, noindex, canonicals and rendered output.
  • Change detection afterwards, so a regression is found in hours instead of in next month's report.

For the last one I built Search Sentinel. It is optional, like all my products.

Ways to work together

Three shapes. Most cleanups start with the first.

There is no price list on this page. Scope follows how much was published and how much access the automation had.

01 / assessment log

Damage assessment

What the automation did, what it cost and what it takes to undo. A fixed piece of work with a clear end, and enough to decide what happens next.

  • A change log of everything that was automated
  • Damage per template, in clicks and revenue
  • A repair and removal plan, ranked on value
Ask for an assessment
02 / cleanup 410

Cleanup project

I run the cleanup together with your team: undo the accidents, triage, fact check and remove. When there is a manual action I write the reconsideration request.

  • Tickets your developers can pick up
  • Removals in batches, checked in the logs
  • Before and after per template
Discuss a cleanup
03 / guardrails [ok]

Guardrails and training

For teams that want to keep automating, or are about to start. Rules, release checks and monitoring, plus a session with the people who run the pipeline.

  • An automation policy with one owner per rule
  • Checks that block a bad release
  • Training on your own pages and data
Set up guardrails

Fair questions. Straight answers.

Whether I am against AI, what Google really penalises, and how long it takes to come back.

Are you against using AI for SEO?

No. I use these tools every day, and my own products come with ChatGPT plugins and MCP servers. What I object to is publishing without review, and giving an agent write access to files that can take a website out of the index.

Does Google penalise AI content?

Not for being written by AI. The spam policy on scaled content abuse is about many pages made to manipulate rankings, "no matter how it's created", and its first example is generative AI used to make many pages without adding value.

One reviewed article written with the help of a model is fine. Forty thousand unreviewed ones are a pattern, and patterns are what gets detected.

Should we just delete everything the pipeline made?

Usually not. Some generated pages answer a real search and convert. Deleting everything removes those too, along with any links they earned. I decide per template with the numbers next to it, and keep what earns its place.

We have a manual action. Can you help?

Yes. The order is: clean up, document what was removed and why, then file the reconsideration request. I cannot promise how fast Google answers or what it decides.

How long does recovery take?

That depends on the cause. A wrong robots.txt rule or a stray noindex is picked up when Google crawls the pages again, so that takes days to weeks. A website that lost ground on quality needs months, and it does not always return to the old level.

Anyone who gives you a date is guessing. An older example of a quality recovery on this blog: how a medical site recovered.

Our agency or a vendor built the pipeline. Is that a problem?

No. I work independently and have nothing to sell in its place. If the pipeline can be made safe, I say so, and the guardrails are written for the people who run it.

How I work inside large organisations is on the enterprise SEO page. The technical side is on technical SEO at scale.

Send me the mess. The pipeline, the pages and the change log, if there is one.

Tell me what was automated, roughly how many pages it published and what happened to traffic since. I will tell you what I would check first.

Independent consultant · working with teams worldwide