According to TheRegister.com, Stanford privacy expert Jennifer King testified before the House Energy and Commerce Subcommittee on Oversight and Investigations that AI developers are exploiting user conversations for model training with little oversight. She revealed there’s no transparency into how companies collect and process data for training, and no requirements for developers to understand their full data pipeline. King emphasized that users are automatically opted into having their data used for training, and companies aren’t proactively removing sensitive information. The situation is worsening as developers run out of English-language public data to scrape, making private user conversations increasingly valuable. Meanwhile, President Trump is seeking to prevent states from introducing AI legislation that could address these privacy concerns.
The privacy nightmare nobody’s talking about
Here’s the thing that should worry everyone: when you’re chatting with an AI assistant about health concerns, relationship issues, or financial problems, you’re probably feeding the very system you’re trying to get help from. King pointed out that we disclose far more personal information in chatbot conversations than in traditional web searches. Think about it – you might share detailed symptoms, personal struggles, or private thoughts that you’d never type into Google. And there’s basically no guarantee that this information gets cleaned or removed before being used to train the next generation of AI models.
The really scary part? Research shows that chatbots can memorize training data. So theoretically, your private health information or personal confession could resurface in someone else’s conversation. Companies like OpenAI are already expanding into browsers and hardware, which means they’re building the infrastructure to collect even more of your data. It’s a gold rush for personal information, and we’re the ones providing the gold without even realizing it.
Why companies want your conversations
As public training data dries up, your private chats become increasingly valuable. King explained that foundation models initially trained on publicly available web data, but that well is running dry. So where do they turn next? Your conversations. And for companies like Meta and Google that already have extensive user profiles, your chatbot interactions become another data stream to monetize.
We’re already seeing companies consider using this data for targeted advertising. Your past shopping experiences could feed into chatbot recommendations. Your health queries might influence the medical ads you see. It’s the same old surveillance capitalism playbook, just with a shiny new AI wrapper. The companies building these systems have every incentive to collect as much data as possible, and very little incentive to protect your privacy.
What happens now?
The testimony before Congress represents a growing awareness of these issues, but the regulatory landscape remains murky. With President Trump opposing state-level AI legislation, there’s a real risk that meaningful privacy protections won’t materialize anytime soon. Companies aren’t required to be transparent about their data practices, and users are automatically opted into data collection by default.
So what can you do? Be extremely careful about what you share with AI chatbots. Assume everything you type could become training data. And support legislation that would require explicit consent for data usage and proper cleaning of sensitive information. The alternative is a future where our most private conversations become public training material for corporate AI systems. Not exactly the future we signed up for, is it?
