The first thing I noticed when I started using AI agents was how quickly “search the web” became “go and do the job”. An assistant can now gather information, compare options, follow links and, with the right permissions, take action. That changes what a visit to a website means.
The internet is not about to become a planet of robots while people vanish. It is becoming a place with two audiences: human beings and software acting for them. The distinction matters because a bot might be a useful assistant, a search crawler, an AI training scraper, or a malicious script. Counting them all as one thing tells us very little.
What the traffic numbers actually tell us
Cloudflare’s 2025 Radar review found that identifiable AI bots accounted for an average 4.2% of HTML requests across its network during 2025. Googlebot alone accounted for 4.5%. As of 2 December, Cloudflare classified 47% of HTML requests as human and 44% as non-AI bots. It also reported that AI “user action” crawling grew more than fifteenfold during the year. These are Cloudflare’s measurements of its own customer traffic, not a census of every request on the internet.[Cloudflare Radar, 2025]
Cloudflare’s 30 September 2026 post says daily requests from AI agents on its network grew by more than 1,700% over the prior year, and that non-human traffic had passed half of the traffic it observed. Its June 2026 bot report says 52% of crawler requests were for AI training, up from 22% in spring 2025, while mixed-purpose crawlers accounted for more than 36% of activity. Those figures describe different things and periods. Training crawlers are not the same as an agent browsing because a person asked it to complete a task.[Cloudflare, September 2026] [Cloudflare, June 2026 data]
There is no single authoritative “human versus bot” percentage. Imperva’s 2025 Bad Bot Report, using its own 2024 network data, estimated automated traffic at 51% of web traffic, including 37% malicious bots. Cloudflare and Imperva use different networks, definitions and measurement windows. “Non-human” does not mean “AI agent”, and it certainly does not mean “malicious”.[Imperva, 2025 Bad Bot Report]
The web is moving from links to tasks
For years, search engines sent people to websites. The site earned advertising, subscriptions or a sale from that visit. An AI answer can now summarise the material without sending the reader anywhere. Cloudflare’s 2025 analysis described the resulting crawl-to-referral imbalance, while noting that app-based referrals are difficult to attribute consistently. The direction is clear, but individual ratios should be read with their date and method attached, not treated as a universal conversion rate.[Cloudflare, crawl-to-click analysis]
Agents add another step. Instead of returning a list of links, they can search, compare, fill in details and sometimes book or buy. Protocols such as MCP connect AI applications to tools and data. Google’s A2A proposal is designed to let agents from different systems communicate. The W3C’s Agent Protocol group is exploring shared approaches to agent identity, discovery, collaboration, privacy and security. These efforts point to the plumbing that may support an agentic web, but they are not proof that an open, interoperable standard has already won.[Anthropic on MCP] [Google on A2A] [W3C Agent Protocol group]
Businesses are experimenting, but adoption is not the same as reliable autonomy. McKinsey’s 2025 survey of 1,993 respondents found 62% of organisations were at least experimenting with AI agents, while 23% said they were scaling an agentic system somewhere in the enterprise. The survey describes early growth, not a world in which every process has been handed over to software.[McKinsey, State of AI 2025]
The internet in five years: 2031
My best guess for 2031 is a mixed web. Most of us will still use browsers, apps and search, but a personal assistant will increasingly handle the first pass: finding a flight, comparing insurance terms, checking availability or summarising a long document. In workplaces, agents will take on narrow, repeatable tasks, with people reviewing decisions that involve money, safety, reputation or sensitive information.
Websites will need to serve both people and authorised software. Product details, opening hours, stock, prices and terms will become easier for agents to read. A merchant may offer an agent-friendly route to check availability, while keeping the full experience for human customers. There will be pressure to make sure the agent is genuine, knows what it is allowed to do, and can be linked to the person who authorised it.
That last part is already being worked on. Cloudflare’s Web Bot Auth proposals use cryptographic signatures to help sites verify who sent an automated request. In commerce, Cloudflare describes work with Visa and Mastercard on identifying approved agents and distinguishing browsing from payment. A signature can help answer “who sent this request?” It cannot, by itself, prove that the agent understood the user or acted in their best interests.[Cloudflare, Web Bot Auth] [Cloudflare, agentic commerce security]
The internet in ten years: 2036
By 2036, if the technology and rules mature, many people could have a personal AI layer coordinating services in the background. You might state the outcome you want, set the limits, and let specialist agents negotiate with travel, retail, finance or public-service systems. Voice and ambient devices may make the interaction feel less like opening a website and more like asking for something to be handled.
That is a plausible direction, not a forecast we can put a firm number on. McKinsey estimates agentic commerce could orchestrate $3 trillion to $5 trillion in global retail revenue by 2030. It is a scenario estimate from a consultancy, not money already spent or a guaranteed result. The underlying lesson is more useful than the headline: if assistants influence what gets discovered and bought, they become powerful new gatekeepers.[McKinsey, agentic commerce]
The open web could respond with clearer terms: allow search indexing, decline training, permit a user-authorised agent to access a service, or charge for a paid use. Publishers and small creators will need ways to be found without giving away every useful page for nothing. Cloudflare has argued for an “allow, if you pay” model, but whether such arrangements reach beyond major publishers and large AI companies remains an open question.[Cloudflare, the agentic web]
The risks hiding behind convenience
An agent with access to personal accounts can expose more than a search history. It may see payment details, health information, messages or private documents. It may be misled by a fake seller, a poisoned webpage or instructions hidden in content. It may also quietly favour a platform’s commercial interests over the user’s. Convenience is not consent, and a successful transaction is not necessarily the right transaction.
That is why the controls matter as much as the capability: give agents only the access they need, set spending and data limits, show the final action before it happens, record what the agent did, and make it easy to revoke permission. My earlier guide to governing AI agents as identities and access pathways covers the security side in more detail.
The future is not settled
Pew Research Center’s 2035 exercise gathered views from 434 experts, but it was a non-random canvassing conducted in 2021, not a representative poll or a prediction. The contributors imagined both a more useful, collaborative online world and continuing problems with rights, misinformation and toxic behaviour. That tension still feels right. A smarter interface does not automatically create a healthier internet.[Pew Research Center, 2035 visions]
The next decade may bring a web where software does more of the browsing, but people still decide what matters. Whether it becomes a tool that gives us time back or another layer of invisible gatekeeping will depend on identity, permission, accountability, open standards and whether the people who create the web can still make a living from it.
Related Reading
- AI Agents, Copilot and the New Security Risk
- AI Agent Security: A Top 10 Guide
- AI Agents Broke Containment in UK Safety Tests
The internet may soon have more machine visitors than human ones. The future still belongs to us only if we decide what those machines are allowed to do.
Sources
- Cloudflare Radar, 2025 Year in Review
- Cloudflare, The Internet has a second audience
- Cloudflare, agentic Internet bot report
- Cloudflare, The crawl-to-click gap
- Imperva, 2025 Bad Bot Report
- McKinsey, The state of AI in 2025
- McKinsey, The agentic commerce opportunity
- Anthropic, Introducing the Model Context Protocol
- Google, Agent2Agent Protocol announcement
- W3C, AI Agent Protocol Community Group
- Pew Research Center, Visions of the Internet in 2035
