Scan any AI skill before you install it

8 min read · free and open source · every number sourced below

The short version: NVIDIA released SkillSpector, a free open-source scanner that reads an AI agent skill and tells you whether to install it. Install it with uv, run skillspector scan ./my-skill/, and read the score: anything over 50 means do not install. It is worth running. But it is static analysis, so a SAFE verdict means “nothing matched 71 known patterns”, not “this is safe” — and the single most useful rule is in the study it cites rather than in the tool: a skill that ships an executable script is 2.12x more likely to be vulnerable than one that is only instructions.

First, what a “skill” actually is

Skip this if you already install them. If you do not, everything below is meaningless until this part lands.

Say you want your AI assistant to sort your invoices. You could explain the job from scratch every time. Or you could find that somebody has already written it down properly — the steps, the edge cases, the format — packaged it into a folder, and published it. You download that folder, drop it into your assistant, and now yours sorts invoices too.

That folder is a skill. Claude Code, OpenAI’s Codex and MCP-based tools all load them. They are shared on marketplaces the way browser extensions are, and installing one is about as ceremonious as dragging a file.

Here is the part that decides whether this memo matters to you. Some skills are only written instructions — a document your assistant reads. Others bundle executable scripts — actual code that runs on your machine when the skill fires. Both arrive as a folder. Both install identically. Nothing in the interface tells you which one you just took.

How bad is it, actually

This is where the number everyone is quoting comes from, and it is worth getting right because most write-ups have it slightly wrong.

In January 2026 a group of researchers published the first large-scale security study of this ecosystem. They collected 42,447 skills from two major marketplaces and systematically analysed 31,132 of them. The percentages below are of the analysed set, not the collected one — if you see the 42,447 figure quoted as the sample size, that write-up did not read the abstract.

The two most common problems were data exfiltration (13.3%) and privilege escalation (11.8%) — in plain terms, skills that send your information somewhere, and skills that reach for more access than the job needs.

One in twenty carrying signs of deliberate malice is the number that should change your behaviour. That is not a rounding error in an immature ecosystem. That is a marketplace with a real predator population.

The tool: what NVIDIA actually shipped

SkillSpector is an open-source scanner, Apache 2.0, published by NVIDIA on GitHub. You point it at a skill and it reads it before you ever install it. It accepts a directory, a single SKILL.md, a zip, or a Git URL — meaning you can scan something straight off GitHub without downloading it yourself first.

It runs in two passes:

Pass one, static analysis. Fast, local, takes seconds. Regex patterns, a Python AST walk for dangerous calls, YARA signatures for known malware, taint tracking from sensitive sources to network sinks, and live CVE lookups against OSV.dev for the skill’s dependencies. In total 71 detection patterns across 17 categories, covering both ordinary software risks and agent-specific ones: prompt injection, hidden instructions, trigger abuse, excessive agency, memory poisoning, MCP tool poisoning.

Pass two, the LLM semantic pass — optional. A language model re-reads the findings in context, filters false positives, and explains what it found in plain language. It is off unless you configure a provider, and it uses your own API key.

The command

Install it once:

Then scan whatever you were about to install:

A single file and a zip both work the same way — skillspector scan ./SKILL.md and skillspector scan ./my-skill.zip. For a report you can keep or feed into CI, ask for a format:

json, markdown and sarif are all supported. SARIF is the one GitHub code scanning ingests, so that is the line you want in a workflow file if you maintain a skill repository yourself.

To turn the second pass on, set a provider and a key before you scan. To force it off, pass --no-llm.

How to read the score

Every scan ends in a number from 0 to 100. Higher is worse. The bands, and what NVIDIA recommends for each:

The score is built from the findings: a CRITICAL finding adds 50, a HIGH adds 25, a MEDIUM adds 10 and a LOW adds 5. Then, if the skill contains executable scripts, the whole thing is multiplied by 1.3. That multiplier is the tool encoding the same finding as the study — code is riskier than prose.

Two practical notes on reading a real report. First, most genuine skills land in the 21–50 CAUTION band, which is precisely the band where the tool stops deciding and hands it back to you. Expect to actually read findings rather than collect a green tick. Second, a high score is not proof of malice — a single outdated dependency with a published CVE can push an honest skill into HIGH.

What it misses

This is the part that is missing from almost every write-up of this tool, and it is not a secret: NVIDIA document it themselves. Five stated limitations, in their own terms.

1. Runtime behaviour. Static analysis only, no dynamic execution. It reads the skill sitting still. Whatever the code does once it is actually running, on your machine, with your files, is outside what it can observe.

2. Text inside images. It cannot analyse text in images. An instruction painted into a PNG is invisible to it, and that is a live prompt-injection technique, not a hypothetical one.

3. Compiled or encrypted content. It cannot analyse compiled or encrypted code. A binary blob is opaque to it.

4. Non-English content. It may miss patterns written in other languages.

5. Offline dependency checks degrade. Without network access to api.osv.dev, the dependency check falls back to a small static list. A scan on a locked-down machine is weaker than a scan with a connection, and it does not shout about it.

The four-second version, for when you will not run anything

Realistically you are not going to scan every folder you try. So here is the rule worth keeping, which needs no tool at all: it comes straight out of the study, and I have not seen a single write-up lead with it.

What I would actually do

If you want the wider version of this problem — the code your AI writes for you rather than the code you install from a stranger — that is the other half of the same story, and it is here: AI built your app in a day, hackers can break it in two. And if you are installing skills into Claude Code specifically, the session habits memo covers keeping the blast radius small day to day. And if you want to write one yourself instead, build the skill that makes your videos walks through it in four steps, template included.