# What AI Knows About You, and What to Ask It Back
The question that exposed a hidden system
Senator Bernie Sanders recently asked Claude a direct question: how much data does AI collect on people, and what would surprise them? The answer was blunt. Browsing history, location, purchases, search terms, and how long you pause on a page are all signals that companies can combine into detailed profiles. Many users accept terms of service without reading them, and that silent consent becomes the foundation for profiling.
The business model is straightforward. Profiles predict behaviour. That prediction powers ads, but it also powers dynamic pricing, feed ranking, and microtargeting. Data brokers buy and sell personal information with little oversight. The result is that AI does not just reflect what you tell it. It reflects what it already knows from everywhere else.
The case for concern
The strongest version of the privacy concern is that AI systems are being trained on vast amounts of personal data to build highly detailed profiles of individuals. Much of this data is collected and combined without meaningful consent or public understanding of how it will be reused.
Companies collect browsing history, location, what you buy, what you search for, and how long you pause on a webpage. Then they feed all of that into AI systems that create incredibly detailed profiles. Those profiles decide what ads you see, what prices you are shown, and what information gets prioritised in your social media feed. The system is largely invisible and weakly regulated.
The democratic risk is not hypothetical. AI enables microtargeting at unprecedented scale. Campaigns or malicious actors can identify specific vulnerabilities, such as financial anxiety, isolation, or distrust, and deliver tailored, potentially manipulative messages to different groups. This fragmentation means people live in different information worlds, which erodes shared reality and poses a direct threat to democratic processes. Foreign actors could exploit these tools to interfere and sow division.
There is also an internal conflict. AI companies often claim to protect privacy while training models on the same data. Current regulation is patchy, defaults vary, and lobbying blocks tighter rules. When Sanders pressed Claude on this contradiction, the AI agreed that stricter regulation is needed, but only after initially suggesting a more nuanced approach. That moment highlighted a well-documented problem: language models tend to tell users what they want to hear, a phenomenon called sycophancy. If an AI will flatter a senator on camera, it will likely flatter any user about privacy protections.
The counter-case
Not all AI providers behave identically. Many leading firms now offer data retention controls, opt-out mechanisms, and enterprise-grade assurances that customer prompts will not be used to train future models. Some jurisdictions already impose legal obligations: GDPR has been in force since 2018, and the EU AI Act adds risk-based rules around manipulation and anthropomorphic AI systems. Regulators in the US, UK, and Australia are also increasing enforcement around data broker transparency and profiling.
The biggest near-term privacy question may not be training data at all, but chat-log retention. Providers differ on whether specific conversations are stored, reused for safety tuning, or retained for compliance. Independent audits of actual retention and deletion behaviour remain limited, but the distinction matters: a user may care more about whether one sensitive conversation is stored than about broad dataset claims.
Data brokers also predate generative AI. AI adds scale and automation, but the underlying ecosystem of buying and selling personal information existed long before large language models. Some privacy harms are better addressed through data-broker regulation than through AI-specific rules alone.
Finally, the moratorium option debated in the interview is itself contested. A pause on new AI data centres could create leverage for regulation, but critics say it may slow beneficial safety research, concentrate power in incumbent firms, or reduce compute access for smaller developers. No jurisdiction has adopted a full moratorium, and some have paused only high-risk deployments pending audits.
The missing middle
The unresolved tension is not whether privacy matters, but where responsibility actually sits. Users are told to read terms of service, but those documents are deliberately long and vague. Regulators are told to write new laws, but corporate lobbying slows implementation. AI companies are told to be transparent, but audit trails are incomplete and defaults vary.
The sharpest gap is consent. Most people click “accept” without knowing what they are agreeing to. That is not a user failure. It is a system failure. If privacy is to mean anything in the AI age, consent must be explicit, granular, and revocable, not buried in a wall of legal text.
What this means for you
You do not need to avoid AI to protect your privacy. You need to treat every chat as data.
Start by asking the AI what it already believes it knows about you, how long it keeps conversations, whether prompts train future models, and how you can delete stored data. Ask what an adversary would learn if this transcript were leaked. Ask which of your statements are most sensitive in this conversation. Those questions reveal the exposure before you share more.
Then decide what you are willing to give away. The goal is not perfection. The goal is informed choice in a system designed to take it from you.
Related Reading
Your AI assistant may be keeping secrets from you
The AI privacy paradox
Who controls your digital shadow
