Pew Research: Over One-Third of New Web Pages Show AI Traces
A study by the Pew Research Center reveals that 35% of web pages created after the release of ChatGPT in November 2022 show signs of being generated or heavily modified by AI. The researchers analyzed a sample of English web pages from the Common Crawl archive using the Open Pangram AI detection tool. This rapid proliferation of AI-generated content could significantly impact the quality of web data, search engine results, and the training of future AI models, which risk "model collapse" if trained on AI-generated data. It also highlights a shift toward a web ecosystem increasingly dominated by automated content. The study found that commercial `.com` domains are roughly ten times more likely to contain AI-generated content compared to government `.gov` and educational `.edu` domains, which sit at around 1%. Additionally, certain stylistic markers like the Oxford comma and specific sentence structures have become noticeably more common in recent web content.
## BACKGROUND
Common Crawl is a nonprofit organization that maintains a massive, open repository of web crawl data, which is widely used by researchers and AI companies to train large language models. Open Pangram is an open-weights AI detection tool designed to identify machine-generated text, though like all AI detectors, it carries a risk of false positives.