I pulled the Cloudflare numbers across Paul's census data website censuseasy.com for the last seven days. For that one site, Cloudflare handled 1,287,768 requests and served 340,177 pageviews.
Roughly four requests arriving for every page a human actually looked at. Some of that is normal, since a page pulls in CSS and images and fonts, but there's a large part that's automated traffic, which is growing exponentially. In the same week Cloudflare blocked 22,471 requests it classified as threats.
If you launch a site today without Cloudflare and with the origin exposed directly to the internet, this is what shows up. Not eventually but within just a few hours.
What actually finds you
The first thing that arrives is the scanner sweep to try to gain access. Automated crawlers walk fresh domains and IP ranges looking for known-vulnerable paths. Requests for /wp-admin on a site with no WordPress. Probes for .env files, exposed .git directories, admin panels on default paths. We covered what happens when one of those probes finds something real, and the answer is that a bot finds it long before a person would.
Then there's data scraping, which is now the loud one. Every price, every listing, every piece of structured data you publish is worth money to somebody who doesn't want to build it themselves. On a data site this can quietly become the majority of your traffic and your bandwidth bill. Paul's lottery data website scratchcheck.com served 70.5 GB worth of bandwidth in a week, and nearly half of that went to things that will never see an ad or click anything. If you're on a small VPS, this can act like a DDOS and take down your site.
Credential stuffing shows up the moment you have a login form. Somebody takes a breach dump from an unrelated service and tries the pairs against you, slowly, from a lot of addresses. Without rate limiting in front, your app is doing the authentication work for every attempt.
And then there's the AI LLM crawlers, which is the newest line item and the one changing fastest.
The AI crawling problem is now a business decision
AI crawlers cost you real bandwidth and CPU, and unlike a search crawler, the historical trade of "you take my content, you send me readers" has broken down. We've written about how AI referrals actually behave, and the TLDR is that some of them do send meaningful traffic as scratchcheck.com receives many AI citations and clicks from it but some take everything and send nothing.
Cloudflare has taken the most aggressive position of any infrastructure company on this. It introduced Pay Per Crawl, letting content owners charge AI crawlers for access rather than choosing between free access and a block. That's since been extended toward paying publishers when their content shapes an AI answer rather than only when a bot fetches a page. And from September 15, 2026, Cloudflare's defaults will block mixed-use crawlers on pages that host ads unless AI companies separate their search crawlers from their AI training and agent crawlers, applying to new customers, new sites, and all existing free accounts.
Whatever you think of that as policy, sitting behind a proxy is what gives you the switch at all. If your origin is exposed, you have a robots.txt file and some wishful hope. Behind Cloudflare you can allow the crawlers that send you readers, block the ones that don't, and change your mind next quarter without touching your application.
It's worth to mention to be careful writing bot rules that catch real crawlers. It's easy to build a challenge rule that also blocks Googlebot and quietly costs you your search presence. Gate those rules on Cloudflare's verified-bot signal so genuine crawlers pass and the impersonators get challenged.
What the setup actually looks like
The part people miss is that pointing DNS at Cloudflare does almost nothing on its own if your server still answers anyone who asks. Anyone who finds your real IP just goes around it, and old DNS records and mail servers leak that IP constantly.
The step that matters is closing the origin. On every site, the firewall should only accept ports 80 and 443 from Cloudflare's published IP ranges, and everything else is denied. SSH password auth is disabled and use SSH keys only. Change the default SSH port from 22 to something random. If you have a static IP, you can also open up the SSH port only to the static IP and nothing else. Setup both hardware and software firewalls. After that, the proxy isn't a suggestion, it's the only door. It also means the DDoS question mostly answers itself, because a flood has to land on Cloudflare's network rather than your box, and their free tier absorbs volumes that would flatten a small VPS.
Two smaller things that also really help out. Turn on caching properly, caching image and static pages can save your origin box from hitting high loads. And trust the real client IP header, because once you're behind a proxy every request appears to come from Cloudflare, which breaks your rate limiting and your logs until you configure it.
Is it actually necessary?
For a hobby project, probably not.
For anything with users, a login, a database or a bandwidth bill, I'd say yes, and I'd put it in the same category as backups. It's not that something bad is definitely coming. It's that the free tier costs nothing, takes about twenty minutes, and the day you need it you cannot install it retroactively. That's the same argument for every unglamorous piece of infrastructure, which is that you buy it before you can prove you needed it.
I'm not especially loyal to Cloudflare as a company and there are real arguments about how much of the web sits behind one provider. Their outages take a lot of sites down at once. But they still offer a lot for just their free tier alone. Just don't put a machine on the public internet with the door open and assume nobody will try the handle, because 22,471 blocked requests in a week across just one site says somebody is always trying it.
Sources
Cloudflare: Introducing pay per crawl - The mechanism letting content owners charge AI crawlers for access instead of choosing only between allowing and blocking them.
TechCrunch: Cloudflare's new policy pushes AI companies to pay for publishers' content - The September 15, 2026 deadline for AI companies to separate search crawlers from AI training and agent crawlers, the default block on mixed-use crawlers for pages hosting ads, and which accounts the new defaults apply to.
Cloudflare: Content Independence Day - The company's stated position on AI crawling without compensation and the shift in default posture for publishers.
Traffic figures are from our own Cloudflare zone analytics across nine sites, for the seven days ending August 7, 2026.
