Cloudflare says artificial intelligence robots in confusion are blocked sites “crawling”

It is claimed that the confusion of the start of AI Search avoids restrictions aimed at preventing web reptiles from AI from accessing certain web sites, according to what it said. Report from Cloudflare. In the report, Cloudflare claims that when the confusion faces a block, the startup will hide its creeping identity “in an attempt to circumvent the site’s preferences.”

The report only adds concerns about the content of the void of confusion without permission, like the company Arrested Previous Paywalls and ignore Robots.txt sites last year. At that time, Aravind Srinivas, CEO of Perplexity Blame the activity On the third party crawl used by the site.

Now, Cloudflare, one of the largest online architecture providers in the world, says she has received complaints from clients who claimed that confusion robots still have access to their websites even after their preference is placed in Robots.txt file on web sites And by creating the rules of the Wall protection on the web app (WAF) to restrict access to AI robots at startup.

To test this, Cloudflare says it has created new areas with similar restrictions against artificial intelligence in the confusion. And I found that the start starting will first try to access the sites by identifying themselves as names for its crawling: “perplexitybot” or “confusion users”.

But if the web site has restrictions on the scraping of artificial intelligence, the Cloudflare claims that the confusion will change its user agent – a part of the information that tells the website any kind of browser and the device you use, or if the visitor is a robot – to cancel the Google Chrome character on MacOS. “Al -Dawarah” that The company does not include In the IP address menu used by its robots.

In addition, Cloudflare claims that confusion changes its independent system networks (ASN), a number used to determine the groups of IP networks controlled by one operator, to overcome the blocks as well. “This activity was observed across tens of thousands of fields and millions of requests per day,” Writes Cloudflare.

In a statement to freedomA confused spokeswoman Jesse Duaire, who was called a “propaganda trick”, adding that “there is a lot of misunderstanding in the blog post.” Since then, Cloudflare has canceled the insertion of a verified confusion, and has offered methods to prevent “ghost crawling” in confusion.