Spam That Outlives the Hack

Proving the Sites Were Clean

SEO spam often cloaks: visitors get the real page, and crawlers get the spam. So before blaming Google, I fetched every homepage on both servers in many ways and compared the answers:

• As a normal browser, as Googlebot desktop and smartphone, and with a Google referer.
• As the Facebook, Twitter, LinkedIn, WhatsApp, Telegram, Slack, Discord, Bing and Bytespider crawlers, and with an empty user agent.
• Through Cloudflare and directly at the origin, with country headers for the US, Thailand, Singapore, Indonesia and Cambodia.

Each fetch printed the status, the final host, the title, the description, the Open Graph and Twitter tags and the canonical URL. On one server that was 480 fetches with zero gambling hits, and every variant got the same status and metadata, so nothing was cloaking by bot or by country.

The rest of the check was read-only too:

• Files: gambling terms in themes, plugins, must-use plugins, drop-ins and webroot files. The only hits were false positives, such as word lists inside a JavaScript library and a PDF hyphenator.
• Databases: options, posts, post meta, SEO fields, terms, approved comments and users, with a time limit per query and per site. The hits were real content, such as an article about the dangers of online gambling.
• robots.txt and sitemaps: every sitemap URL, followed recursively. The sitemaps were clean. Four robots.txt files were not, as described below.

So the sites were clean, and people were seeing what Google had stored during the attack.

Why Google Kept the URLs

During the compromise, the attacker had served spam pages to Googlebot under URLs like /?g=a1b2c3d4 or /shops/anion/s1p3i3r1i4f8. After the cleanup those pages were gone, but most of the URLs still answered 200.

WordPress ignores query parameters it does not know and shows the homepage. So /?g=a1b2c3d4 returned the clean homepage with status 200, and to Google the URL was alive. It kept the URL and the spam title it had stored for it. On one site a single parameter shape had 87,000 requests answered with 200 in five weeks.

Real people still clicked those results. About 160 visits on one server came in from Google or Bing on spam URLs, and on the other server the same visits showed up as junk traffic in Google Ads and Analytics.

Mapping the Spam from Access Logs

I could not ask Google for a list of what it had indexed, but the access logs showed what its crawler kept asking for. A script read about five weeks of NGINX logs, compressed ones included, and counted Googlebot and Bingbot requests by URL shape, per site and per hostname.

Server	Spam URLs still crawled	Sites affected
First	About 300,000	7
Second	About 660,000	10

The shapes repeated across sites, which made them possible to match:

• A one-letter parameter with an interleaved letter and digit code: /?t=o4b3f0u0s7c8a8t5.
• Zero to two words, then a code: /shops/anion/s1p3i3r1i4f8e2r0853.
• A number or URL-encoded Korean text ending in .html: /?ID=2009.html.
• Paths ending in .shtm, .shtml or the misspelled .sthml.
• Numbered spam sitemaps: /sitemap-subitem-2090.xml, /sitemap-child-12.xml.
• Shell control traffic such as ?ARRAY= and old loaders like /wper.php?sitemap-308.xml.

A 410 Rule Tested Against Real Logs

404 tells Google a page is missing, perhaps for now. 410 Gone says it was removed on purpose, and Google drops those URLs faster. NGINX can answer before PHP runs, so each request also costs almost nothing. The core of the rule on the second server was two lines per site:

# /etc/nginx-rc/extra.d/<app>.location.main-before.spam-410-generic.conf
if ($args ~ "^[a-z]=([a-z][0-9]){3,}[a-z0-9]*$") { return 410; }
if ($uri ~ "^/([a-z]+/)?([a-z][0-9]){4,}[a-z0-9]*(\.shtml?)?/?$") { return 410; }

The first server needed eleven patterns, generated from the same shape list the log scanner used, so the rule and the analysis could not drift apart.

A rule like this can take a real page down, so before installing it I ran it against every request from the same five weeks, about 4.7 million on the first server:

• No real URLs matched. No page, asset, sitemap or internal link, and none of the URLs that carried gclid, gbraid, utm_*, fbclid, msclkid or _gl, about 43,000 on one site alone. A broken ad landing page would have cost the client money.
• Only spam and scanners. On clean sites, the only matches were vulnerability scanners probing paths that already returned 404.
• It found gaps. The first version missed the URL-encoded Korean ?ID= spam. The test showed the shape, and I added it before going live.

The install script checks the NGINX config, reloads, and then compares every homepage and a homepage with ad parameters against their answers from before. If anything answers differently, it removes the rule and reloads again. After the install, every spam sample answered 410, through Cloudflare and as Googlebot.

Telling Google

A 410 only works once Google requests the URL again, and for some of these URLs that could take months. Two Search Console tools speed it up.

Temporary “gone” sitemaps
A sitemap usually lists pages you want indexed, but Google crawls whatever it lists. I published a gone-urls.xml per site with every spam URL that the rule turns into 410, so Google would recrawl them soon and see the answer. Google accepts at most 50,000 URLs per sitemap file, so each one is a sitemap index pointing to parts of 45,000 URLs or fewer. They come down after about six weeks, while the 410 rule stays.

Removals by prefix
Search Console can hide every URL under a prefix, such as https://example.com/?t=. That is powerful and dangerous, so a script derived the prefixes from the crawled spam and kept one only if it covered at least 20 spam URLs and zero real ones. It checked every prefix against every sitemap URL and every non-spam URL that had answered 200 since the cleanup, and it never proposed a bare domain or a bare ?.

Not robots.txt
I did not block the spam paths in robots.txt. A blocked URL is never fetched, so Google would never see the 410 and would keep the URL in its index.

1. 01
   Fetch every site as every bot, through Cloudflare and at the origin
2. 02
   Count the spam URLs crawlers still request, by shape
3. 03
   Test the 410 rule on five weeks of logs Zero real pages or ad URLs may match
4. 04
   Install with automatic rollback, then verify through Cloudflare
5. 05
   Tell Google
   • Gone sitemaps Google recrawls and sees 410
   • Removals by prefix Only prefixes with zero real URLs

robots.txt Without a Bad Word

My first sweep reported every robots.txt as clean, because none of them contained a spam word. Four of them were still written by the attacker:

User-agent: *
Crawl-delay: 3
Sitemap: https://example.com/sitemap.xml

It looks harmless, but on that site /sitemap.xml had been the attacker’s spam sitemap, and on another the file listed loader URLs such as wzy.php?sitemap.xml. Two clues gave them away: each file was born within a minute of the webshells on the same site, and it pointed crawlers at the old spam sitemap instead of the site’s real sitemap index.

Cloudflare caches robots.txt too, for up to 30 days here. One site had a clean file on the server while Cloudflare still served the attacker’s version, so I now compare robots.txt at the origin and through Cloudflare, and purge the cache after replacing it.

Every Hostname Counts

One client site also answered on two alias domains. Most of its spam, about 67,000 URLs, had been indexed under an alias, not under the main domain. A Search Console property only covers its own host, so the cleanup there needed a new Domain property, verified by a DNS TXT record, before any sitemap or removal could be submitted.

The log scan reported spam per hostname for this reason. The gone sitemaps were written per hostname, except for a www host that redirects everything to the main domain, whose URLs went into the main file.

Limitations

• It takes weeks. Google drops 410 URLs as it recrawls them, and the “Not found” count in Search Console rises for one to three weeks while it does.
• Other caches keep their copy. Link preview tools and other crawlers keep old titles until they fetch the page again.
• The client has to submit it. Search Console belongs to the site owner, so the sitemaps and removals were a checklist for them, not something I could finish from the server.
• New shapes need new patterns. The rule only matches shapes seen in the logs. The daily report still checks what each homepage serves to Googlebot.

Lessons

Search results lag behind the cleanup
Removing the malware ends what the site serves. Google keeps what it has already seen until it fetches each URL again and gets a clear answer.

Return 410 for spam URLs
WordPress answers 200 for almost any query string. Until those URLs return 410, Google treats them as live pages.

Build the rule from the logs
The access logs told me which shapes to match, and they were also the test for whether the rule was safe.

Check what the cleanup tools skip
A word-based scan will pass a malicious robots.txt, and a server-side check will not see an old copy cached at Cloudflare.

List every hostname
An alias domain gets its own crawl history, its own Search Console property and its own share of the spam.

Most of this work happened in the logs and in Search Console rather than on the server, and it took as long as the malware cleanup.