Here is a scenario I run into more than you'd expect. A founder builds an app with an AI tool, everything looks great, and buried in their project is a package the AI installed with total confidence. The catch: that package was never supposed to be there. It either doesn't exist the way the AI thought it did, or worse, someone malicious registered that exact name knowing AI tools would recommend it.

This is the problem of AI hallucinated packages, and it is one of the sneakiest security risks in AI-built apps right now. It doesn't show up in your demo. It doesn't throw an error you can see. It just sits in your project, quietly, until it becomes a way for someone to steal your data or hijack your app.

I see the conditions for this almost every week when I review AI-built apps. So let me explain what a hallucinated package actually is, why AI tools confidently invent them, why this opens the door to a real attack, and a concrete checklist you can use to audit your own app's dependencies even if you've never written a line of code.

What a hallucinated package actually is

First, a quick plain-English definition. A package (sometimes called a dependency or a library) is a chunk of pre-written code that your app pulls in so it doesn't have to build everything from scratch. Want to send email, process a payment, or format a date? There's a package for that. Modern apps use dozens or even hundreds of them.

These packages live in big public warehouses. For JavaScript apps it's npm. For Python it's PyPI. When your app needs a package, it downloads it from one of these warehouses by name. The name matters enormously, because the name is the only thing your app uses to decide what to install.

A hallucinated package is one the AI recommends that doesn't really exist, or doesn't exist the way the AI claims. The AI tells you to install react-auth-helper or stripe-easy-checkout, you run the command, and one of two things happens:

  • The install fails because the package genuinely isn't there. Annoying, but harmless.
  • The install succeeds, because someone else noticed AI tools keep suggesting that name and registered it themselves. Now you've installed code written by a stranger, and it runs inside your app.

That second case is the dangerous one. The package looks legitimate. It installs cleanly. Your app runs. And you have no idea you just invited an outsider's code into your project.

Why AI tools confidently recommend packages that don't exist

This part trips people up, so let me be clear about the root cause. AI coding tools don't actually know what packages exist. They are not looking anything up in a live catalog when they answer you. They are predicting what code looks right based on patterns from everything they were trained on.

So when you ask an AI to "add login to my app," it generates code that resembles the thousands of login examples it has seen. Part of that pattern is importing a helper package. The AI fills in a name that sounds plausible, like auth-utils or secure-login-js, because that's the shape of names it has seen before. It is not checking whether that exact package is real. It is just confident, because confidence is what these models do.

AI tools don't look up packages. They predict names that sound right, which means a fake name and a real one look identical coming out of the machine.

Researchers have a name for this when it happens with packages: package hallucination or "slopsquatting." Studies have found that a meaningful share of AI-recommended packages simply don't exist. And here's the part that makes it dangerous instead of just annoying: the AI tends to hallucinate the same fake names over and over, because it's drawing from the same patterns every time.

That predictability is the whole problem. If an attacker knows that AI tools frequently suggest a package called fast-email-sender, they can register a real package with that exact name and stuff it full of malicious code. Then they wait. Every founder whose AI suggests that name and runs the install command gets the attacker's code, automatically.

This is the same blind-spot problem I keep coming back to. Using AI to build, and then using more AI to check the build, repeats the same gaps. The model that hallucinated the package name is not going to reliably catch its own hallucination.

Why this is a real supply-chain attack, not a hypothetical

The phrase "supply-chain attack" sounds abstract, so let me make it concrete. A supply chain is everyone whose code ends up in your product. When you install a package, you are trusting not just that package but everything it installs, all the way down. One package can pull in twenty others.

A supply-chain attack means someone poisons one link in that chain, and the poison flows downstream into every app that trusts it. You didn't write the bad code. You didn't even choose it directly. The AI chose it for you, and you ran the command.

Here is what malicious package code can actually do once it's running inside your app:

  • Steal your secret keys. Your payment processor key, your database password, your email API key. Many AI-built apps keep these in places that running code can read.
  • Read or exfiltrate your user data. If the malicious code runs on your server, it can quietly send your users' information somewhere else.
  • Open a back door. Some malicious packages install a way for the attacker to get back in later, even after you think you've cleaned up.
  • Hijack your build or deploy process. The worst ones inject themselves into the code you ship, so every visitor to your app gets affected.

None of this announces itself. There's no popup that says "malware installed." The app runs normally in your demo, which is exactly why it slips past non-technical founders. The damage happens in the background, and you often don't find out until your keys are abused or a service flags suspicious activity on your account.

This connects directly to the other security gaps I see in AI-built apps. If your secrets aren't locked down, a malicious package has a feast waiting for it. I wrote more about that broader pattern in the hidden cost of vibe-coding debt, and a poisoned dependency is one of the most expensive forms that debt can take.

How to spot the warning signs yourself

You don't need to be a developer to do a first pass on this. You need to know where to look and what's normal. Here are the signs that make me want to take a closer look at an app's dependencies.

  • The AI installed a lot of packages you don't recognize. A small app shouldn't need a hundred dependencies. If the list is huge, something is being over-pulled.
  • Package names that sound too convenient. Real, well-known packages have boring, established names. A name like easy-secure-auth-helper-pro that perfectly matches what you asked for is a yellow flag, because that's exactly what an AI invents and an attacker squats.
  • A package with almost no usage or history. Legitimate popular packages have millions of downloads and years of history. A package with 40 downloads and a publish date from last month deserves suspicion.
  • You can't find the package documented anywhere reputable. If searching the name turns up nothing, or only a bare listing with no real project behind it, be careful.
  • The install command came straight from an AI chat and you ran it without checking. That's the most common path to this problem.

The single most useful file to know about is your project's dependency manifest. In a JavaScript app it's package.json, and the full locked list is package-lock.json. In Python it's usually requirements.txt. These files list every package your app depends on, by name. They are the map of your supply chain.

Your dependency audit checklist

Here's the practical part. I'll give you a checklist you can work through, and I'll be honest about which steps you can do alone and which ones are worth handing to a real developer.

Step 1: Find your dependency list

Open your project and find package.json (JavaScript) or requirements.txt (Python). Look at the list of packages. You don't need to understand the code, just read the names. This is your inventory.

Step 2: Run an automated security scan

Both major ecosystems have a built-in command that checks your packages against a database of known vulnerabilities. This is free and takes one line.

# For JavaScript / Node apps
npm audit

# For Python apps (after installing pip-audit)
pip-audit

This won't catch a brand-new malicious package that nobody has reported yet, but it will flag known bad ones immediately. Run it. Read what it reports. If it lists "critical" or "high" issues, that's your priority list.

Step 3: Check the suspicious names by hand

For any package name you don't recognize, look it up on the official site:

  • JavaScript: search the name on npmjs.com.
  • Python: search the name on pypi.org.

Check three things: how many downloads it has (popular packages have a lot), when it was last updated (active projects get maintained), and whether there's a real project behind it (a linked code repository, a readme, an actual description). A package that fails all three is a package I'd pull out.

Step 4: Verify the package actually matches what your app needs

Sometimes the AI installs a real package, but it's the wrong one, or an obscure clone of a famous one with a slightly different name. This is typosquatting: stripe versus stipe, react-router versus react-route. Read the names carefully and compare them to the well-known originals.

Step 5: Lock down your secrets regardless

Because you can't be 100 percent certain you caught everything, reduce the blast radius. Make sure your secret keys aren't sitting in your code where any running package can grab them. If you suspect anything was compromised, rotate your keys, which means generating new ones and retiring the old. This same secrets discipline matters for your login system too, which I covered in why your AI-built auth is probably broken.

Step 6: Get a human to review the high-stakes parts

Here's my honest line. You can absolutely do steps one through five yourself, and doing them puts you ahead of most founders. But confirming that a questionable package is actually safe, or tracing what a suspicious dependency does once it's running, is real detective work. That's the part where guessing wrong is expensive, because the cost is your users' data and your reputation.

That's exactly the kind of thing I get pulled in for: going through an AI-built app's full dependency tree, identifying anything fake, squatted, or simply unnecessary, and replacing it with something trustworthy without breaking the app you already have.

Keeping it from happening again

Once you've cleaned up, a few habits keep this from creeping back in. Don't blindly run install commands an AI hands you. When you add a package, take ten seconds to look it up first. Keep your dependency list as small as you reasonably can, because every package you add is one more link in your chain that someone could poison. And review your dependencies on a regular rhythm, the same way you'd review your bills. This is part of the ongoing care every live app needs, which I get into more in who maintains your AI app after launch.

The reassuring truth is that hallucinated packages are findable and fixable. They feel scary because they're invisible, but they leave a clear trail once you know where to look. A short audit today is far cheaper than a breach later.

If you've built something with AI and that list of dependencies is a mystery to you, you don't have to keep wondering whether one of them is a problem. I review AI-built apps every week, check the full supply chain, and tell you plainly what's safe, what's fake, and what needs to go. If you'd like that peace of mind, let's talk about what your app is actually running.

Cover photo by Pavel Danilyuk on Pexels.