Your AI Tools Are Not Hacked Yet. The Next Attack Won’t Warn You.

Philip Hall
-
0
Your AI Tools Are Not Hacked Yet. The Next Attack Won’t Warn You.

News Analysis  |  27 June 2026

I spent the morning testing a Microsoft 365 Copilot tenant for a client last week. The IT team told me Copilot was locked down, that Microsoft had handled it. The tenant was clean. What we found instead was a policy gap big enough to drive a data exfiltration truck through. That is the real story this week: vendors are shipping AI features faster than security teams can keep up, and attackers are already inside the gap.

81%

of security teams have no visibility into AI-generated code in their own repositories, according to industry surveys cited in the 2026 Enterprise AI Security Index.

The Screen Is Deceiving You

Tuesday’s NCSC warning was blunt: hostile states are now linked to roughly three-quarters of cyber attacks on UK critical infrastructure, and AI is the accelerant. At the same time, the US White House published updated AI cybersecurity guidance with a 30-day implementation window that puts federal agencies on a hard deadline for vulnerability scanning, patch coordination, and exposure management. The message from both sides of the Atlantic is the same: get the basics right, move fast, or assume compromise.

The most practical threat this week is privilege drift in AI-connected tools. Microsoft 365 Copilot, Google Workspace assistants, and GitHub Copilot all operate with implicit user permissions. If a user can access an email thread, the AI can read it. If the user can download a patient record or a contract, the AI can summarise it. That is not a design flaw. It is a trust model that security teams have not yet caught up with. The Cycode 2026 AI security report confirms that public AI security incidents rose 56.4% from 2023 to 2024 and are still climbing.

What This Means

You do not need a new budget to reduce the worst exposure. Start with identity and access for AI integrations. Review which applications can call your AI tools. Check whether service accounts used by Copilot or similar agents are over-provisioned. Turn off any AI connector that has no current business case. Second, treat AI-generated content as untrusted input. Code, emails, and documentation created by AI can carry hidden instructions that trigger when humans or other AI systems read them. That is not science fiction. OWASP Top 10 for LLM Applications lists prompt injection as the number one AI-native attack vector, and major vendors are now shipping runtime protections. If you are not running those protections, you are exposed.

AI security is not a product you buy. It is a governance decision you make every time you connect a new data source to an AI tool.

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

ChatGPhish: How ChatGPT Turned Into a Phishing Machine

Philip Hall
-
0
ChatGPhish: How ChatGPT Turned Into a Phishing Machine

ChatGPhish: How ChatGPT Turned Into a Phishing Machine

I’ve been saying it for months now: the way we use AI tools is creating security holes we don’t even see yet. This week brought another proof point, and it’s not some theoretical risk buried in a research paper. It’s a phishing bug called ChatGPhish, and it affects the way users get answers from web summaries inside ChatGPT.

Here’s what happened. A vulnerability was discovered in how ChatGPT renders summarized web pages, specifically in its Markdown conversion process. When ChatGPT gives you a quick summary of a web page, it strips away a lot of the original HTML and replaces it with its own condensed format. That summarization is useful, but it also means cues you rely on to judge a link’s trustworthiness, like the real URL or page branding, can get lost or misrepresented. Attackers figured out they could upload or reference malicious pages that, when summarized by ChatGPT, would produce phishing content that looked like legitimate information delivered by the AI itself.

Why does this matter? Because most users have learned to trust AI outputs as filtered, cleaner versions of the web. If the AI gives you a link, you’re less likely to scrutinise it than if you opened a random web page yourself. That trust is exactly what ChatGPhish targets. The attack surface isn’t a single company server anymore. It’s every AI-assisted search, every research assistant, every tool that condenses the web on your behalf.

For Australian businesses, this should raise specific alarms. Many teams rely on AI assistants to speed up research, draft client summaries, or scan partner websites. If the AI silently redirects intended destinations to phishing pages, one bad summary becomes a compromise entry point. Traditional security controls often don’t cover this path because the AI acts as an intermediary between the user and the actual web request.

The practical takeaway: treat AI-generated links the same way you would treat any link from an untrusted source. Hover over them whenever possible. Where you can, require human verification before users act on AI-supplied content. Better yet, segment how front-line staff access AI summarisation tools, especially when those tools pull from external web sources.

What’s most frustrating here is the cycle keeps repeating. First cloud apps, then email, then messaging platforms, then AI assistants. Every new convenience layer becomes a new exploitation layer. The attackers aren’t inventing new techniques as fast as we’re creating new surfaces for them to attack.

Trust in the tool is not a security control. The moment AI summarisation becomes a default path for users, it becomes a default target for phishers.

Related Reading:
– https://philiphall.com/ai-safety-net-holes-security-2026
– https://philiphall.com/ai-dethroned-stolen-passwords-hackers-verizon-2026
– https://philiphall.com/ai-just-broke-the-19-year-record-heres-what-it-means-for-your-business

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

Scammers Are Now Hosting Malware on chatgpt.com. Yes, the Real One.

Philip Hall
-
0
Scammers Are Now Hosting Malware on chatgpt.com. Yes, the Real One.

The rise of Scammers Are Now Hosting Malware is reshaping how we think about technology and security. I’ve spent twenty years telling people to check the URL before they click anything. If it says google.com or microsoft.com, you’re probably fine. If it’s russian-bank-login-totally-real.xyz, you run.

That rule just died.

Security researchers at Push Security have uncovered a campaign they’re calling LLMShare, and it’s one of the craftiest things I’ve seen this year. Attackers are using ChatGPT’s own content sharing feature to host malicious pages. The URL starts with chatgpt.com. The padlock icon is there. The domain is legitimate. Everything looks right.

It’s completely fake.

How the Attack Works

Here’s the playbook. It’s simple, which is exactly why it works.

Someone searches Google for “ChatGPT.” They see a sponsored ad at the top of the results (attackers are running Google Ads campaigns targeting these keywords). They click it. Instead of landing on a normal ChatGPT page, they’re taken to a shared ChatGPT conversation, sitting on the real chatgpt.com domain.

The page shows what looks like a legitimate outage notice. “We’re experiencing high traffic right now,” it says. “Our website is temporarily unavailable due to a large number of users. Download our desktop app to continue.”

Sounds plausible. ChatGPT does go down. They do have a desktop app. What’s the harm?

The “download” button takes victims to a site called openew.app, which is dressed up to look exactly like OpenAI’s official download portal. From there you get a choice: Windows or Mac. Both versions contain malware. The Windows version checks if it’s running in a virtual machine or sandbox first (researchers call this cloaking), meaning antivirus scanners often see nothing wrong. If it’s a real computer, the payload executes.

Why This Matters More Than a Standard Phish

We’ve all seen fake login pages before. What makes LLMShare different is the domain. It’s not chatgpt-support.com or openai-download.net. It’s chatgpt.com. The real one. Verified certificate. Green padlock. The whole trust infrastructure that we’ve spent two decades building into browsers is now working against us.

The attackers exploited ChatGPT’s HTML rendering capability. They wrote a prompt that generates a custom HTML and CSS page, published it as a shared conversation, and got a legitimate chatgpt.com/s/ URL out of it. Push Security noted that if you look closely, you can actually see the “Show code” and “Remix with ChatGPT” buttons on the page. It’s literally just ChatGPT rendering a malicious webpage.

This isn’t a vulnerability in the traditional sense. OpenAI’s systems are working as designed. The attackers are simply using the feature more creatively than the designers imagined.

It’s Not Just ChatGPT

This is part of a broader trend. Earlier this year, attackers ran Google Ads directing Claude users to shared Claude conversations containing malicious installation instructions. Other campaigns abused shared ChatGPT and Grok conversations to run ClickFix attacks, tricking victims into pasting commands into their terminals that installed malware.

Claude’s Artifacts feature has been used the same way. Every AI platform with a content sharing feature is now a potential malware distribution vector. The platforms built these features for collaboration and sharing. Criminals saw a trusted domain with user-generated content and built a scam delivery network on top of it.

This is what happens when platforms move faster than their threat models. AI companies are shipping features at breakneck speed. The security implications of those features often get considered after the fact.

What You Should Actually Do

First, the obvious stuff. Only download ChatGPT from openai.com. If you’re ever on a page that looks like ChatGPT but is asking you to download something, check the actual URL carefully. If it says chatgpt.com/s/ followed by a random string, you’re looking at someone’s shared conversation, not an official page.

Second, and this is important for IT teams: update your security awareness training. The old “check the domain” advice is no longer sufficient when attackers can host content on the same domain as the legitimate service. Your staff need to understand that chatgpt.com can serve malicious content, just like drive.google.com or dropbox.com can.

Third, endpoint protection that detects VM and sandbox evasion is not optional anymore. The LLMShare malware specifically checks whether it’s being analyzed before executing. If your security tools can’t flag that behaviour, you’re flying blind.

Fourth, organisations should seriously consider blocking or monitoring shared AI conversation links in corporate environments. I know that sounds heavy-handed, but when the attack surface includes legitimate domains that your proxy and firewall inherently trust, you need compensating controls.

Trusting a domain is not the same as trusting everything on it. The browser doesn’t know the difference between OpenAI’s outage page and a criminal’s ChatGPT conversation. You have to.

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

A Single Dodgy Character Just Broke Millions of AI Agents. Here’s What You Need to Do.

Philip Hall
-
0
A Single Dodgy Character Just Broke Millions of AI Agents. Here’s What You Need to Do.

The landscape of Single Dodgy Character Just continues to shift in ways that demand attention. I have been watching the AI agent space for a while now. Everyone is racing to plug large language models into everything they own. Email, calendars, code repositories, clinical databases, industrial control systems. It is the Wild West out there, and honestly, the security posture of most AI infrastructure makes your average IoT camera look like Fort Knox.

This week we got the bill.

Security researchers at X41 D-Sec discovered a vulnerability in Starlette, the Python framework that underpins FastAPI. You have probably not heard of Starlette, but you have definitely used something built on it. It has 325 million downloads a week. FastAPI, vLLM, LiteLLM. Essentially every piece of Python AI serving infrastructure sits on top of this thing.

The bug, tracked as CVE-2026-48710 and branded “BadHost” by the researchers, is embarrassing in its simplicity. Drop a single manipulated character into the HTTP Host header, and you bypass path-based authorization entirely. That is it. No complex exploit chain. No memory corruption. No cryptographic weakness. Just … a character.

The researchers scanned the internet to see what was actually exposed. What they found should make any CISO lose sleep.

What Is Actually Exposed Right Now

Biopharma companies with their clinical trial databases wide open. Identity verification systems leaking live personally identifiable information. Industrial IoT systems that let attackers SSH straight through corporate bastion hosts. Email servers where you could read, send, and delete any message in any mailbox. HR platforms exposing full candidate pipelines including background check data. Document management systems. Cloud monitoring dashboards revealing AWS topology and distributed traces. Cybersecurity companies with their own asset inventory and live vulnerability scanners exposed.

But the scariest target class is MCP servers. These are the Model Context Protocol servers that AI agents use to connect to third-party services. Think of them as the AI agent’s hands. They store credentials. Email credentials, calendar access tokens, database logins, API keys for payment processors. One BadHost exploit on an exposed MCP server, and you do not just own the AI agent. You own every single service it is connected to.

Why This Keeps Happening

I have written about this before on this site. The AI tooling ecosystem is shipping faster than it can secure. Starlette gets 325 million downloads a week and not a single person in the entire chain, from framework maintainer to application developer, thought to validate the Host header. Not one.

This is not really a Starlette problem. It is an ecosystem problem. We are building skyscrapers on foundations that nobody inspected. When the NSA issued its first warning about MCP security last week, this was exactly the kind of thing they were worried about. Now we have a live, trivial-to-exploit vulnerability affecting essentially the entire Python AI stack.

What You Need to Do Right Now

First, check your Starlette version. If it is anything before 1.0.1, you are vulnerable. The fix shipped last Friday. Upgrade immediately. However, assume attackers are already scanning for unpatched systems, because they are.

Second, run the free scanner at mcp-scan.nemesis.services. Tell it your domain. It will show you if any of your exposed services are reachable.

Third, audit every MCP server you run. If they are storing credentials for email, calendars, databases, Slack, GitHub, or anything else, those credentials are one Host header away from compromise. Rotate every single one of them after you patch.

Fourth, review your authentication middleware. Starlette’s routing uses the actual request path, but request.url.path (which your auth logic reads) can be manipulated. The two no longer match. That is the entire bug. Any middleware that makes authorization decisions based on request.url.path needs to be rewritten to use the raw request path instead.

Fifth, put your AI infrastructure behind a properly configured firewall. BadHost is trivial to exploit, but it requires network access. If your MCP servers and model endpoints are not reachable from the public internet, you just bought yourself time to patch properly.

This one is bad. Not because the exploit is sophisticated. It is not. It is bad because the blast radius is enormous and the fix, while simple, requires every team running Python AI infrastructure to actually do something. In an ecosystem where most teams do not even know what version of Starlette they are running.

“The speed at which AI infrastructure is being deployed has completely outpaced basic security hygiene. We are building skyscrapers on foundations nobody inspected. BadHost is not the last one. It is just the first one someone bothered to look for.”

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

300,000 ChatGPT Accounts Got Hacked Last Year. Here’s What It Means for Your Business.

Philip Hall
-
0
300,000 ChatGPT Accounts Got Hacked Last Year. Here’s What It Means for Your Business.

The 300,000 ChatGPT Accounts Got Hacked space is moving fast, and this latest development proves it. Here’s a number that should wake you up: over 300,000 ChatGPT account credentials got swiped by infostealer malware last year. That’s not a typo. Three hundred thousand.

IBM’s X-Force team dropped their 2026 Threat Intelligence Index this quarter, and while the headline numbers are bad enough, the underlying message is worse. AI tools have become the same kind of target as your corporate Salesforce or your HR platform. The attackers don’t care that it’s an AI assistant. They care that it’s another SaaS login with access to company data.

AI Isn’t Reinventing Attacks. It’s Supercharging Everything We Already Sucked At.

This is the part most coverage gets wrong. Nobody at IBM is saying AI invented some terrifying new attack vector. What they’re saying is a lot more practical and a lot more alarming: AI is making existing attacks dramatically faster, and that changes the economics of defence.

Attacks that start by exploiting public-facing web applications jumped 44% in a single year. Forty-four percent. Most of those vulnerabilities required no authentication at all. Zero credentials. The attacker just scans, finds the hole, and walks in.

Mark Hughes, IBM’s global head of cybersecurity services, put it plainly: “Attackers aren’t reinventing playbooks, they’re speeding them up with AI. The core issue is the same: businesses are overwhelmed by software vulnerabilities. The difference now is speed.”

Let me translate that. Your security team was already drowning in CVEs before AI showed up. Now the attackers can scan for those unpatched systems, identify the exploitable ones, and launch an attack faster than your team can finish their morning coffee.

Your ChatGPT Login Is Now a Corporate Liability

Those 300,000 stolen ChatGPT credentials aren’t just a number. What happens when an attacker logs into your employee’s ChatGPT account and reads the conversation history? What proprietary data got pasted in there? What internal strategy documents, what code snippets, what customer details?

The threat goes deeper than data theft. A compromised AI account lets attackers manipulate outputs, inject malicious prompts, and potentially pivot to other systems. IBM’s team found that once an attacker compromises an AI platform, they move laterally to other data stores 60% of the time.

And here’s the kicker: 97% of organisations that had an AI-related breach lacked proper AI access controls. Not a typo. Ninety-seven percent. Most companies are deploying AI tools faster than they’re securing them, and the attackers have noticed.

Supply Chains Are Getting Hammered. Again.

If you’ve been in this industry for more than five minutes, you’ve heard the supply chain security sermon a hundred times. Well, IBM’s data says it’s gotten nearly four times worse since 2020. Large supply chain and third-party compromises have almost quadrupled.

The new twist is AI-assisted code generation. Developers are pumping out features faster than ever using AI coding tools, and some of that code is making it into production without proper security review. The trust relationships built into modern CI/CD pipelines give an attacker who compromises one component a free pass to everything downstream.

The Ransomware Numbers Nobody’s Talking About

Active ransomware groups surged 49% in 2025. Nearly half again as many gangs operating as the year before. But here’s what changed: these aren’t large, sophisticated operations. Most are small crews running low-volume campaigns, getting in and out fast.

Why? Because the barriers to entry have collapsed. Leaked ransomware tooling is everywhere. Playbooks are well documented. AI handles the translation, the reconnaissance, the target research. You don’t need a nation-state’s resources to run a ransomware operation anymore. A motivated teenager with a ChatGPT subscription and some stolen credentials can cause real damage.

What You Should Actually Do About It

I know these articles tend to end with vague platitudes about “investing in security.” Let me be specific.

  • Treat AI tools like enterprise SaaS. If your staff are using ChatGPT, Claude, or any AI platform, those accounts need the same access controls as your CRM. Multi-factor authentication, session monitoring, conditional access. No exceptions.
  • Patch your internet-facing applications. Forty percent of incidents started through vulnerability exploitation, and most required no authentication. This is the cybersecurity equivalent of locking your front door.
  • Audit your supply chain. If a vendor has access to your data or your pipeline, you need to know their security posture. Not their marketing PDF. Their actual posture.
  • Get an AI governance policy written. Sixty-three percent of breached companies didn’t have one. Don’t be one of them.

The IBM report confirms what a lot of us in the industry have been saying for two years now. AI hasn’t created a new class of threats. It’s just made the existing ones work at a speed that traditional security operations can’t match. The companies that adapt fastest will survive. The ones waiting for a silver-bullet AI defence product will become the next data point in next year’s report.

Security leaders need to shift to a more proactive approach, using agentic-powered threat detection and response to identify gaps and catch threats before they escalate.

Mark Hughes, Global Managing Partner for Cybersecurity Services, IBM

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

The AI Safety Net is Full of Holes: What 2026 Taught Us So Far

Philip Hall
-
0
The AI Safety Net is Full of Holes: What 2026 Taught Us So Far

The conversation around AI Safety Net is Full of Holes has reached a critical point. Look, I’m not here to be the fun-police. I love AI as much as the next bloke. But we’ve hit a point in 2026 where the hype has officially outrun the brakes, and the brakes weren’t even attached to begin with.

I was reading through the latest Experian and White & Case reports this morning, and the numbers are bloody terrifying. We’re talking over 8,000 major data breaches in just the first half of last year, with 345 million records floating around in the wild. If you think the “big guys” have this under control, you’re dreaming. In fact, 69% of people don’t believe banks or retailers are ready for what’s coming.

The shift we’re seeing right now isn’t just about a clever hacker in a hoodie. It’s about Autonomous AI Agents. These things are self-operating bots that can execute complex attacks without a human even lifting a finger. It’s like leaving the front door unlocked and finding out the burglar is an invisible robot that can pick locks at light speed.

Here’s the reality: your data is being used as training material for the very tools that will eventually be used to scam you. We’ve seen a massive spike in “Synthetic Identities” – AI-created profiles that look so real they can bypass most standard verification checks. One in four millennials has already been hit by identity theft this past year. That isn’t a statistic; it’s a crisis.

Also, regulators are finally waking up, but they’re creating a patchy mess. From the DOJ bulk data rules to Missouri and Maryland passing their own “Online Data Privacy Acts,” businesses are drowning in compliance while the hackers are just getting more efficient. If you’re a business owner, you can’t just tick a box and hope for the best anymore.

So, what should you actually do?

First, stop feeding the beast. If your staff are using AI tools without a clear policy, they’re likely leaking your trade secrets and customer data into a public model.

Second, get serious about “Privacy-Enhancing Technologies.” If you aren’t looking at quantum-resistant encryption yet, you’re already behind. The attackers are already using AI to find vulnerabilities in current standards.

Third, verify everything. If a video call from your “boss” asks for a transfer or sensitive files, call them back on a different number. Deepfakes aren’t just for viral TikToks anymore; they’re the new phishing.

We’re in an era where cyberattacks aren’t just about stealing your credit card; they’re about manipulating digital reality itself. Don’t be the low-hanging fruit.

> “AI is evolving at breakneck speed, and cybercriminals are the early adopters. If you isn’t using AI to defend your data, you’re bringing a knife to a gunfight.”

### Related Reading
* [Your Staff are Feeding AI Tools 18,000 Terabytes of Company Data](https://philiphall.com/your-staff-are-feeding-ai-tools-18000-terabytes-of-company-data-most-bosses-have-no-idea/)
* [The NSA’s Warning on AI Agent Security](https://philiphall.com/nsa-mcp-ai-agent-security-warning-2026/)
* [Why Stolen Passwords are Still King](https://philiphall.com/ai-dethroned-stolen-passwords-hackers-verizon-2026/)

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

AI-Generated Political Attack Videos Are Now Mainstream. Heres Why That Terrifies Security Pros

Philip Hall
-
0
AI-Generated Political Attack Videos Are Now Mainstream. Heres Why That Terrifies Security Pros

AI-Generated Political Attack Videos Are is a topic I have been following closely, and the developments keep coming. This week, a 22-second AI-generated video showed something that should make every cybersecurity professional stop and think. The clip, posted on Truth Social, depicts a prominent figure grabbing a late-night talk show host and physically throwing him into a dumpster on stage. The crowd cheers. The figure dances. The video is convincing enough to make you look twice.

It was created with generative AI. However, it went viral within hours.

Now, set aside the politics for a moment. What matters here is not who posted it or who the target was. What matters is that a synthetic media clip depicting a real person in a physically impossible scenario was produced, distributed, and consumed by millions without any kind of warning label, content provenance marker, or verification mechanism baked into the distribution chain. That is a watershed moment for AI-generated misinformation.

The Intelligence Community Saw This Coming

Avril Haines, the US Director of National Intelligence, has been warning for months that AI can generate “seemingly authentic” deepfake content capable of influencing audiences and spreading false narratives at scale. This video is a textbook demonstration of exactly that capability.

The technology behind these clips is not cutting-edge. It is openly available. The same tools that can make a convincing deepfake of a CEO’s voice for a business email compromise attack can generate a video of a political figure doing something they never did. The same diffusion models that power creative applications can be repurposed for targeted attacks against individuals.

Why This Is a Security Problem, Not Just a Political One

From a cybersecurity standpoint, the proliferation of AI-generated video content creates several urgent problems:

Synthetic media undermines trust in all digital evidence. When a convincing deepfake of any public figure can be created in minutes, the default position shifts from “seeing is believing” to “nothing is real.” This erosion of trust is exactly what threat actors exploit in disinformation campaigns.

Detection is still playing catch-up. Deepfake detection tools exist but they are not deployed at the distribution layer. Social media platforms, messaging apps, and news sites have no consistent requirement for AI-generated content labelling. The technology to detect synthetic media exists, but the infrastructure to apply it at scale does not.

The barrier to entry keeps dropping. The video that went viral this week was not produced by a state-level actor with unlimited resources. It was produced with consumer-grade AI tools. As the cost of generating convincing synthetic video approaches zero, the volume of this content will explode.

What This Means for Australian Organisations

Australian businesses and government agencies are not immune to this trend. Deepfake technology is already being used in Australian contexts, from voice cloning scams targeting executives to fabricated evidence in social engineering campaigns. The same tools that produce viral political content are being repurposed for targeted attacks against Australian organisations.

The Australian Signals Directorate and the Cyber Security Centre have flagged synthetic media as an emerging threat vector. If your organisation has not yet updated its incident response plan to account for AI-generated disinformation, this week’s events should be a wake-up call.

The Path Forward

Content provenance standards like C2PA (Coalition for Content Provenance and Authenticity) are gaining traction, but adoption is voluntary and uneven. Regulation is being discussed in multiple jurisdictions but has not materialised in any meaningful form. Detection technology continues to improve but faces an asymmetric battle against generative models that improve even faster.

For now, the best defence is scepticism and verification. Treat any video or audio clip that confirms your existing biases with extra scrutiny. Verify through multiple independent channels before acting on synthetic media content. However, push the platforms you use to implement mandatory AI content labelling.

The era where a video could be taken as proof is over. We are now in the era where every piece of digital media must be treated as potentially synthetic until proven otherwise. Security professionals who adapt to this reality will protect their organisations from what comes next.

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

The NSA Just Issued Its First Formal Warning About AI Agent Technology. Your IT Team Needs to Read It.

Philip Hall
-
0
The NSA Just Issued Its First Formal Warning About AI Agent Technology. Your IT Team Needs to Read It.

Every time I think I have seen it all with NSA Just Issued Its, something new emerges. I’ve been watching AI agent technology move from developer toy to corporate backbone over the past 18 months, and the security conversation has been almost entirely missing. This week, the NSA made that conversation unavoidable.

On May 22, the US National Security Agency published its first formal cybersecurity guidance specifically targeting Model Context Protocol (MCP), the underlying technology that lets AI assistants like Copilot, Claude, and ChatGPT connect to your files, calendars, databases, and business systems. The NSA’s conclusion is blunt: organisations are deploying this technology faster than they understand what they’re handing it access to.

What MCP Actually Is

Most people using AI tools at work have no idea MCP exists. It’s the plumbing. When your AI assistant books a meeting, reads a contract, queries your CRM, or writes code that runs against a live database, MCP is what connects the AI model to those systems. Anthropic created it, and it’s now embedded in production workflows at financial institutions, law firms, and software companies globally.

The NSA describes MCP as “the de facto standard” for AI-driven services. That’s actually the problem. A protocol that connects AI models to everything has become standard before anyone built a security model for it.

What the NSA Found

The advisory lists six categories of risk, and none of them are theoretical. Weak or missing authentication. Poor approval workflows. Insecure data handling. Missing audit logs. Session hijacking vulnerabilities. Prompt injection attacks that let malicious content hijack what the AI does on your behalf.

The NSA notes that real-world exploits have already been documented: “poorly secured MCP tools used to access private information or run harmful commands.” This is not a warning about what might happen. It’s a warning about what is already happening.

Research published by Noma Security earlier this month found that one in four MCP servers exposes AI agents to arbitrary code execution risk. A typical enterprise now runs over 100 high-risk tools connected to its agents. Most of those connections have no version pinning, meaning a silent update to a malicious version could run in production before anyone notices.

The Speed Problem

The core issue isn’t that MCP is fundamentally broken. It’s that the adoption timeline has compressed what should have been a multi-year security maturation process into a matter of months. The NSA’s own words: MCP’s rapid adoption has “outpaced the development of its security model.”

Companies wanted the productivity gains. The AI tools delivered them. The security conversation got deferred. Now the NSA is the one having it, which means the deferral period is over.

I’ve seen this pattern before. A useful technology gets adopted at speed. The security infrastructure builds slowly behind it. The gap between the two is where attackers live. With AI agents, that gap is enormous because the tools are highly capable, deeply connected, and often running with admin-level permissions that nobody explicitly approved.

The Shadow AI Multiplier

This gets worse when you factor in shadow AI. The Verizon DBIR published last week found that employee use of unapproved AI tools tripled in a single year, jumping from 15% to 45%. Most of those tools connect via MCP or similar protocols. Most of those connections aren’t in any IT inventory. Most of the data flowing through them isn’t being logged.

The NSA is warning about sanctioned MCP deployments. The real exposure is the unsanctioned ones that nobody is watching at all.

What to Actually Do About It

The NSA’s recommendations are practical and worth implementing now, regardless of how your AI tools are deployed:

  • Audit what your AI tools connect to. Most organisations can’t answer this question. Start there.
  • Apply least privilege. If an AI assistant needs to read emails, it doesn’t need write access to your database. Scope the permissions.
  • Separate sensitive systems. High-risk data environments should have extra barriers before any AI automation touches them.
  • Log everything. AI agent activity needs audit trails. If you can’t see what the agent did, you can’t detect when it was misused.
  • Validate tool inputs. Prompt injection is a real attack class. Systems that ingest untrusted content into AI workflows need filtering.
  • Pin MCP server versions. Silent updates from a poisoned package are a documented attack vector. Don’t rely on whatever the latest version happens to be.

The NSA is not saying stop using MCP. They’re saying stop treating it as invisible infrastructure that doesn’t need the same scrutiny you’d apply to any other system that touches sensitive data. That’s a reasonable ask.

The Broader Shift

We’re at an inflection point where AI tools have graduated from being interesting experiments to being core operational infrastructure. The security conversation needs to make the same jump. Governance frameworks that were written before agentic AI existed don’t cover this. Procurement processes that check a security questionnaire box but never ask what MCP servers the AI connects to don’t cover this either.

The NSA publishing a formal advisory is a signal that the intelligence community considers this a live, active risk surface. That should carry weight with every CISO and every board that has signed off on AI tooling without asking hard questions about what it’s connected to.

The most dangerous thing about AI agents isn’t what they can do. It’s that nobody in most organisations knows what they’re doing right now.

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

One VS Code Extension. One Developer. 3,800 GitHub Repositories Gone.

Philip Hall
-
0
One VS Code Extension. One Developer. 3,800 GitHub Repositories Gone.

Every time I think I have seen it all with One VS Code Extension. One, something new emerges. I’ve been saying for years that the biggest security risk in most organisations isn’t the firewall or the servers. It’s the developer’s laptop. This week, GitHub proved me right in the most spectacular way possible.

On May 20, Microsoft-owned GitHub confirmed that a hacking group called TeamPCP had stolen data from roughly 3,800 of its internal repositories. The attack vector? A single employee installed a poisoned VS Code extension. That’s it. One bad plugin on one machine, and suddenly a hacking crew is rifling through thousands of GitHub’s own code repositories and offering the stolen data for sale on a cybercrime forum with a starting price of $50,000.

GitHub says there’s no evidence of customer data being impacted, and I’m inclined to believe them on that narrow point. But let’s not lose sight of what actually happened here. The company that hosts the world’s software got hacked through the exact same supply chain weakness that’s been exploited repeatedly in 2026, and they had zero visibility into what extensions their developers were running.

TeamPCP Have Been Busy

Here’s what makes this more alarming than a one-off incident. TeamPCP isn’t some new crew stumbling onto a technique. In 2026 alone they’ve compromised Trivy, Checkmarx, Bitwarden CLI, TanStack, and now GitHub. All through developer tooling. All using the same basic playbook: get malware onto a developer’s machine via a trusted tool, then use that foothold to reach further into the network.

Mackenzie Jackson from Aikido Security put it plainly: “Developer workstations are the number one target in supply chain attacks right now. Most security teams still have zero visibility into what extensions or packages are on their developers’ machines, or how recently they were published. That’s the blind spot these attacks keep walking through.”

That’s the real story here. Not just GitHub. The blind spot is everywhere, in every company with developers using VS Code, which is basically all of them.

Why VS Code Extensions Are a Genuine Crisis

VS Code extensions aren’t sandboxed. They have full access to everything on the machine they run on: credentials, SSH keys, cloud API keys, environment files, tokens sitting in memory. A developer’s laptop is essentially a master key to your entire infrastructure, and VS Code extensions get handed a copy of that key the moment they’re installed.

The VS Code marketplace has hundreds of thousands of extensions. Microsoft does review them, but the review process has never been designed to catch sophisticated supply chain attacks where a legitimate extension is later poisoned via a compromised publisher account or a dependency update. The attacker doesn’t need to create a new malicious extension from scratch. They just need to get access to one that developers already trust.

GitHub hasn’t named the specific extension involved. That omission matters. Without that information, every developer using VS Code right now has no idea whether they’ve been exposed to the same poisoned tool.

What You Actually Need to Do

If you’re responsible for a team of developers, or you are a developer, here’s what this week’s GitHub breach tells you to action:

  • Audit every VS Code extension across your developer machines. You need to know what’s installed, who published it, when it was last updated, and whether the publisher account shows any signs of compromise.
  • Treat developer machines as high-value targets in your threat model, not just endpoints. The credentials and tokens sitting on a developer’s laptop can give an attacker more access than a successful phishing attack on an executive.
  • Rotate credentials and secrets regularly, and treat any secret that has lived on a developer machine as potentially compromised if that machine is breached. GitHub rotated critical secrets immediately after detection, which is good practice, but reactive rotation is always worse than proactive rotation.
  • Consider restricting VS Code extension installs via policy to an approved list. Yes, developers will complain. That’s fine. The alternative is what happened to GitHub.

GitHub is investigating and has promised a full incident report. When that comes out, the specific extension name should be disclosed. Until then, treat your VS Code extension inventory as an open security question.

The Supply Chain Is the Attack Surface Now

What TeamPCP is doing in 2026 is the logical evolution of supply chain attacks. Rather than targeting one organisation directly, they’re targeting the tools that developers at hundreds of organisations use every day. Compromise Trivy (used for container vulnerability scanning) and you potentially have access to every CI/CD pipeline that runs it. Compromise a VS Code extension with millions of installs and you have a foothold on millions of developer machines simultaneously.

We wrote recently about a worm that hit 160+ npm packages including OpenAI. That was the same group, the same technique. The GitHub breach isn’t a surprise. It’s a continuation.

The security perimeter isn’t your network edge anymore. It’s your software supply chain, and most companies have no idea what’s in it.

“A single VS Code extension on one employee’s machine was enough to get access to 3,800 internal GitHub repositories. Most security teams still have zero visibility into what extensions or packages are on their developers’ machines. That’s the blind spot these attacks keep walking through.” – Mackenzie Jackson, Aikido Security

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

Your Staff Are Feeding AI Tools 18,000 Terabytes of Company Data. Most Bosses Have No Idea.

Philip Hall
-
0
Your Staff Are Feeding AI Tools 18,000 Terabytes of Company Data. Most Bosses Have No Idea.

Few topics in technology right now are as important as Your Staff Are Feeding AI. I had a conversation with a CFO last week who was proud of his company’s AI policy. “We’ve banned ChatGPT,” he told me. I asked him if his team used Grammarly. He said yes, of course, everyone does.

That’s the problem right there.

New data from Zscaler’s ThreatLabz 2026 AI Security Report makes for uncomfortable reading if you run any kind of business. Researchers analysed 989.3 billion AI and machine learning transactions across enterprise networks in 2025, and what they found should be on every board agenda this week.

Employees at enterprise companies transferred 18,033 terabytes of data to AI apps last year. That’s a 93% jump in a single year. The biggest recipient wasn’t ChatGPT. It was Grammarly, with 3,615 terabytes of corporate text flowing into its systems. ChatGPT came in second at 2,021 terabytes. Those 410 million Data Loss Prevention violations tied to ChatGPT alone included attempts to share social security numbers, source code and medical records.

Let that sink in. 410 million violations. In one year. From one tool.

The tools your people are using every day have quietly become, as the Zscaler report puts it, “the world’s most concentrated repositories of corporate intelligence.” Grammarly reads your emails, your proposals, your legal documents, your client strategies. Every time someone pastes text into it and hits “improve,” that text goes somewhere.

This isn’t a criticism of Grammarly or ChatGPT specifically. The problem is the governance gap, or rather, the total absence of one. Enterprise AI usage grew 91% year-on-year across more than 3,400 applications. Engineering teams account for nearly half of all AI usage (48.9%). IT teams handle another 31.8%. These are the people touching your most sensitive systems and codebases.

Meanwhile, separate research from Hadrian, drawn from data across 300-plus organisations, found that 99.5% of security alerts are false positives. Security teams are drowning in noise, unable to find the 0.47% of genuinely exploitable issues buried in thousands of irrelevant notifications. The average time to remediate a critical vulnerability is four days. Some stay open for four months. Not because nobody noticed. Because teams couldn’t distinguish the real threats from the background static.

So here’s where we are: AI adoption is accelerating at machine speed, governance is moving at human speed, and security teams can barely see what’s real. Attackers, by contrast, are using AI for reconnaissance, for exploit chaining, for automated lateral movement. They know exactly where to strike. Defenders are still reading tickets.

What you should actually do about this

Start with a simple audit. List every AI tool your team uses, including the ones embedded in existing software (AI writing assistants baked into Microsoft 365, for instance, or AI features inside your CRM). You probably don’t have a complete list. That’s the point.

Second, implement Data Loss Prevention policies before your employees paste something they shouldn’t. Most enterprise security platforms support DLP rules for common AI endpoints. It’s not a perfect solution but it closes the most obvious doors.

Third, if you’re in finance, healthcare, legal, or any regulated industry, you need to categorise what data is allowed to touch external AI systems. Source code, client contracts, financial projections and patient records should have explicit policies attached. “We don’t use AI with sensitive data” is not a policy. It’s a wish.

Finally, ask your security team how they’re handling alert triage. If the answer is “manually,” you have a problem. The math doesn’t work when you’re looking at thousands of alerts per day and 99.5% of them are noise. Automation and prioritisation tools aren’t optional extras anymore.

The Zscaler report notes that AI governance “has transitioned from a policy discussion to an immediate operational necessity.” The simpler version: your staff are feeding your business intelligence into AI systems you didn’t approve, at a scale you probably haven’t measured, and most of those systems have been found to contain critical vulnerabilities. Every single enterprise AI system in the Zscaler research had at least one. Every one.

That’s not a technology problem. It’s a leadership problem.

“The biggest risk going into 2026 isn’t that organisations lack security tools. It’s that they no longer know which threats are real while attackers know exactly where to strike.”

Rogier Fischer, CEO, Hadrian

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

1...19202122Page 20 of 22