Aditya Sharma

Cloudflare Scrape Shield: What It Does and What It Doesn’t

On this page, 8 sections

Cloudflare Scrape Shield is a group of settings that Cloudflare’s documentation describes as meant to protect your site’s content. In the current docs it lists exactly two features: Email Address Obfuscation and Hotlink Protection. Neither one stops a bot from reading your pages, so it is not a general anti-scraping shield, whatever the name suggests.

I read the Scrape Shield index, both feature pages and Cloudflare’s AI bot pages on 11 October 2026, then tested both features against my own Cloudflare-fronted site on 12 October 2026. One result surprised me: Cloudflare rewrote a share-by-email link on one of my posts that contains no email address at all.

Below: what each feature does, how to test it from a terminal, what the documentation leaves out (marked “not documented”), and where to look if what you actually want is to control AI crawlers.

What the docs list under Scrape Shield

FeatureWhat Cloudflare’s page saysPage last updated
Email Address ObfuscationHides email addresses on your pages from bots while keeping them visible to humans.3 August 2026
Hotlink ProtectionPrevents your images from being used by other sites, which can reduce origin bandwidth.3 August 2026

The Scrape Shield index page itself was last updated on 16 April 2026 and lists only those two. Older guides, including a 2023 post of mine, describe a third setting, Server-side Excludes. The current Scrape Shield index does not list it. So whether Server-side Excludes still exists, or was removed, is not documented in the docs I could reach. Check your dashboard rather than trusting an old tutorial.

Email Address Obfuscation, tested

Cloudflare says it enables this feature automatically when you sign up. When it is on, Cloudflare replaces email addresses in your HTML with links like [email protected], and injects a small deferred script, email-decode.min.js, that reveals the real address in the browser. Humans see no change. A scraper that only reads raw HTML sees the encoded version.

The docs list exceptions. Addresses are not obfuscated inside HTML tag attributes other than the href of an a tag, nor inside script, noscript, textarea, xmp or head tags, nor on pages that are not HTML.

To keep one address readable, wrap it in <!--email_off--> and <!--/email_off-->. You can also turn the feature off in the dashboard under Security settings, or per hostname with a configuration rule.

To see whether it is active on your site, count the encoded links on a page that has an email link. This one-liner passed bash -n and I ran it on 12 October 2026:

#!/usr/bin/env bash
# usage: emailcheck.sh https://example.com/some-page/
curl -s "$1" | grep -o '/cdn-cgi/l/email-protection#[0-9a-f]*' | wc -l

Run against my post on protecting emails from bots, it printed 1. The homepage showed none. Here is what that one match was. The post has a social share row with an “Email” button. Its href in the served HTML was /cdn-cgi/l/email-protection#e5da96..., and the page carried Cloudflare’s email-decode.min.js script.

I decoded the hex string with the first byte as an XOR key. It contained only ?subject=How%20to%20Protect%20Emails...&body=https%3A%2F%2Fadityaarsharma.com%2F.... In other words, a mailto: link with a subject and a body but no address.

That fits the docs in one sense: the exception list says the href of an a tag is processed. Whether Cloudflare is meant to rewrite a mailto: link that contains no address is not documented on the page I read.

Whether the share button still works for a visitor with JavaScript switched off, I did not test. If you run share buttons, click yours in a private window and check the link target before assuming anything.

For the wider picture of keeping addresses away from scrapers in WordPress, see how to protect emails from scraping in WordPress.

Cloudflare says Hotlink Protection checks image requests and denies those whose HTTP referer does not include your site’s domain and is not blank. It supports five extensions: gif, ico, jpg, jpeg and png. Visitors on your own domain still see the images. You can exempt files by putting them in a directory named hotlink-ok, or exempt a path with a configuration rule.

The sentence that matters for scraping is this one from the docs: it has no impact on crawling, though it stops images from being displayed on sites such as Google Images, Pinterest and Facebook.

So Hotlink Protection is about other sites embedding your image files from your server, not about someone downloading your content. By the rule as written, a request with a blank referer is not denied. That reading is my inference from the documented rule, not a statement from Cloudflare.

Here is a script that requests one image URL with three referers: a foreign site, none, and your own. It passed bash -n.

#!/usr/bin/env bash
# usage: hotlink.sh https://example.com/wp-content/uploads/2026/01/photo.jpg
IMG="$1"
for ref in "https://other-site.example/page" "" "https://example.com/"; do
  curl -s -o /dev/null -w "referer='${ref}' -> %{http_code}\n" -e "$ref" "$IMG"
done

I ran it on a JPEG on my own site. All three requests returned 200, including the one with a foreign referer. That tells me Hotlink Protection is not blocking that URL on my zone.

I did not open the dashboard to confirm the toggle, so I am only claiming what the response showed. If yours is on, you should see a non-200 on the first line and 200 on the other two.

What Scrape Shield does not do

  • It does not stop text scraping. Nothing on the two feature pages describes blocking requests for page content or detecting bots. The email feature changes how addresses appear in HTML, and the image feature checks the referer on image requests.
  • It does not control AI crawlers. That sits in other products. Cloudflare’s AI Crawl Control page, last updated 14 August 2026 and marked as available on all plans, describes monitoring which AI services access your content, allow or block rules for individual crawlers, and tracking robots.txt compliance. Pay per crawl is listed there as a private beta.
  • It does not replace verifying who is calling. A scraper can send any referer or user agent. To check a crawler’s identity by IP, see verify the real GPTBot vs a fake user agent.

The Block AI Bots page lets you set a policy per behaviour: Search, Agent and Training. Each has three actions: block on all pages, block on pages with ads, or allow. It also describes a “Block AI bots” setting as deprecating on 15 September 2026, plus new defaults for new domains starting that date.

The page was last updated on 1 July 2026 and written in the future tense, so I cannot tell you from the page whether those changes have taken effect. Open Security settings on your own zone to see what you actually have.

Cloudflare’s controls and your robots.txt work together, not as substitutes. robots.txt is a request that compliant crawlers follow, and the standard says it is not access authorization. I wrote the WordPress side of that in how to block AI crawlers in WordPress.

Not documented, so not claimed

  • Whether Server-side Excludes still exists (the current index does not list it).
  • Whether Email Address Obfuscation is meant to rewrite mailto: links that contain no address.
  • Any measured effect of the injected decode script on page speed. The docs say it is deferred and does not block rendering. I did not benchmark it.
  • Any success rate against real scrapers. Cloudflare’s pages describe behaviour, not effectiveness, and I did not run a scraper against my site.

Questions people ask

Is Cloudflare Scrape Shield free?

The two feature pages I read do not state a plan requirement, and I did not check pricing, so that is not documented in what I read. The one availability statement I found is on the AI Crawl Control page, which says it is available on all plans.

Will email obfuscation stop spam completely?

Cloudflare says it hides addresses from bots, and the exceptions above show where addresses stay readable. It covers addresses in the HTML Cloudflare serves. An address leaked elsewhere, such as in an attribute, a script, a JSON response or a PDF, is outside it. I have no measured spam rates, so I will not put a percentage on it.

Can Hotlink Protection break my images on social sites?

Cloudflare says it prevents images from being displayed on sites such as Google Images, Pinterest and Facebook, and documents the hotlink-ok folder and configuration rules for images you want shared. If you rely on link previews, test one before turning it on.

Do this today

Run the email check on a page that has a mailto link, and the hotlink script on one of your images. Write down what each returned.

Then open Security settings for your zone and compare it with the output, because the output is what visitors and bots get. If your goal was to limit AI crawlers, go to AI Crawl Control and your robots.txt instead.

Resources

Tell me where I am wrong

Your email is not published and I do not add it to any list. Corrections with a source are the ones I act on fastest.

More in Cyber Security

Every piece in Cyber Security

  1. How Many Passkeys Can a YubiKey Hold? 25 or 100 by Firmware Yubico's firmware matrix says 100 passkeys on firmware 5.7 and later, 25 on 5.0 to 5.5. What counts, how to check your key, and… Cyber Security 6 min
  2. Bitwarden vs Google Password Manager: What the Docs Actually Say I read both vendors' documentation: how each encrypts your vault, who holds the key, what is free and what Bitwarden Premium costs. Here is… Cyber Security 6 min
  3. Vaultwarden vs Bitwarden Self-Hosted: Which Server to Run Vaultwarden is unofficial and light. Bitwarden's own server is official, now with Lite. What each project documents, its audits, and who should run which. Cyber Security 9 min
  4. Verify the GPTBot IP Address: Real Crawler or Fake User Agent A user agent proves nothing. Check the IP against OpenAI's published JSON, and Googlebot with reverse plus forward DNS. Scripts included, each one run. AI 9 min
  5. Lost YubiKey: What to Do in the First Hour and the First Week There is no central off switch for a lost YubiKey. You remove it service by service. Here is the order, what Google and Bitwarden… Cyber Security 6 min
  6. Is Chrome’s Password Manager Safe? What Google’s Own Pages Say Chrome encrypts and breach-checks saved passwords, but Google's own security blog names a gap with malware on Windows. What the pages say and what… Cyber Security 6 min