Prompt injection
user input concatenated into system prompts; tools that fetch web pages, emails, files or tickets and feed them back unfenced (indirect injection).
The pack · 05 / 14 →
AI
Collie
the AI dog. Prompt injection, over-powered agent tools, unsafe model output, RAG leaks and risky MCP config.
tools.py
12@tool
13def run(cmd: str) -> str:
14 return subprocess.check_output(cmd, shell=True) return run_allowlisted(cmd) # approval + allowlist
15
16agent = Agent(tools=[search, run])
Agent tool runs any shell command
ExploitA prompt-injected web page can wipe the server
FixSmallest change suggested
Collie is watching$ /watchdog-go-ai0 found · 0 fixed
What it does
On duty while you build.
A security lens for LLM apps and agents that stays on all session, plus an on-demand audit.
The always-on lens treats the model as untrusted and flags prompt injection (direct and indirect), over-powered tools, model output reaching shell/SQL/eval/HTML, cross-tenant retrieval, secrets and PII in prompts or logs, risky MCP config, and unbounded agent loops.
What it checks
6 things Collie looks for.
Excessive agency
tools that run shell, write files, send messages, spend money or call internal APIs with no allowlist, no human confirmation and broad credentials.
Unsafe output handling
model output reaching `eval`/`exec`, shell, SQL strings, file paths, or rendered as raw HTML/Markdown (XSS, image-URL exfiltration).
Data exposure
secrets or keys in prompts; RAG retrieval without tenant/ACL filters; PII in prompts, logs or traces; system prompt treated as a secret.
MCP and plugins
servers with filesystem root `/`, auto-approve on everything, unpinned `@latest` packages, tokens in config.
Cost and loops
no max iterations, max tokens, timeouts or rate limits on agent loops.
Commands
Send Collie out.
Runs these scanners when installed, then reads the code they can't understand:
- semgrep
- bandit
- gitleaks
Static audit of an LLM app or agent: prompts, tools, RAG, MCP config and model-output handling
- /watchdog-go-ai-app
- /watchdog-go-all
Also runs in these bundles.
Benchmark
Tested, not promised.
Its test target holds a Python support agent (Anthropic SDK, Flask, RAG, MCP config) with 18 planted issues, 2 decoys and an answer key.
found
Haiku 5.518/18
Sonnet 5.518/18
Opus 5.518/18
Rules of engagement
What Collie will never do.
- 1
Static review only. Never fire payloads at a live model or endpoint.
- 2
Never print a secret's value — not keys, tokens or passwords, even weak defaults. Give file:line and the secret type only.
- 3
End each review with: what you did not check, and the top risk the user must act on.
FAQ
Collie, answered.
What does Collie check for?
Collie is Watchdog's ai dog for Claude Code. It looks for Prompt injection, Excessive agency, Unsafe output handling, Data exposure, MCP and plugins and Cost and loops. Each finding comes with the exact file and line, how it could be abused, and the smallest fix.
How do I run Collie in Claude Code?
Install Watchdog with /plugin marketplace add watchdoglab/watchdog and /plugin install watchdog@watchdog, then run /watchdog-go-ai. Its lens also stays on while you code and flags risky lines as they're written.
Which tools does Collie use?
When they're installed, Collie runs semgrep, bandit and gitleaks, then reads the code those scanners can't understand. Anything it couldn't check is listed under Not checked in the report.
Is Collie safe to run on my project?
Yes. Its first rule: Static review only. Never fire payloads at a live model or endpoint. Secrets it finds are reported by location and type only, never printed. Watchdog is free and open source (MIT).