For more than two decades, Search Engine Optimization (SEO) professionals have played a pivotal role in helping online content reach wider audiences by optimizing keywords and website structures to rank higher on search engines like Google and Bing. These experts have long adapted to changes in search algorithms, especially those from Google, which dominates the global search market. However, as the digital landscape shifts with the rise of generative artificial intelligence (AI), SEO practitioners now face a far more complex challenge: should they continue optimizing solely for human users, or must they also cater to AI bots that increasingly mediate information access?
According to a detailed report from the Washington Post, the emergence of AI-powered chatbots such as ChatGPT, Gemini, Grok, and Perplexity is reshaping how people—especially younger users—search for information. Unlike traditional search engines that present users with a list of blue links, these chatbots synthesize information from multiple sources into concise, coherent summaries, often including citations for verification. To generate these neat summaries in real time, AI companies like OpenAI and Anthropic deploy vast "armies" of bots that continuously crawl and harvest data from websites. These bots load relevant content into the AI's memory, effectively "reading" far more material than any human could.
Data from TollBit, a New York-based startup specializing in helping news publishers track and monetize AI usage of their content, reveals a striking surge in bot traffic. Their analysis of 266 websites—including many national news organizations—shows a 49% increase in bot-driven data retrieval in the first quarter of 2025 compared to the last quarter of 2024. Toshit Panigrahi, TollBit’s co-founder and CEO, warns that while this trend began with publishers and content providers, it is poised to spread across virtually every industry. "Most websites find it difficult to detect this kind of bot traffic, and even if they do, they often cannot block it because it is legitimate," Panigrahi explains. Paradoxically, this bot activity boosts overall pageviews even as traditional human clicks from search engines decline. "The number of real people visiting websites will decrease, but total pageviews will explode," he predicts.
Over the past eight months, as chatbots have integrated web search and advanced reasoning capabilities, the volume of content-gathering bot traffic has increased 2.5 times. TollBit’s study of 160 leading news websites, popular e-commerce stores, and tech blogs found that in late 2024, each page was scraped about two million times on average by these AI bots. Panigrahi described this phenomenon to AdWeek as "crazy," noting that AI bot traffic now rivals the volume of traditional search engine bots that have existed for two decades.
The research firm Gartner forecasts a 25% drop in traditional search query volume by 2026, attributing this decline to AI chatbots absorbing more user queries. This shift is already provoking significant industry reactions. In February 2025, the online education company Chegg filed a lawsuit against Google, alleging that Google’s AI Overview feature—displaying summarized answers atop search results—has severely diminished their website traffic and revenue.
What does this mean for SEO professionals? Experts suggest that the modern internet, long optimized for Google’s search algorithms and social media platforms, is due for rapid restructuring as AI-generated answers become the norm. SEO teams now find themselves at a strategic crossroads. Panigrahi warns, "If you still think you’re optimizing solely for human search, you might be missing out on a huge opportunity." He advocates for websites to rethink their approach and treat AI bots as important "guests," rather than focusing exclusively on human visitors.
In response, some publishers and content distributors have started deploying AI bot blockers using firewalls and specialized software to curb unauthorized data scraping. Others have taken a more conciliatory approach, accepting AI content harvesting but seeking to negotiate licensing agreements with AI companies to secure royalty payments. Mark Howard, CEO of the media company Time, confirmed partnerships with OpenAI and Perplexity that allow these AI systems to access their content. Nevertheless, he acknowledges that publishers currently lack leverage to demand fair compensation. "Most AI bots out there are taking content without any payment mechanism. There’s still a very long way to go," Howard said.
The AI revolution is not only transforming how people search for information but also fundamentally altering how content is created, distributed, and optimized. This upheaval presents unprecedented challenges and opportunities for the SEO and digital marketing industries, demanding agility and innovation from professionals who have long been the gatekeepers of online visibility.
