The Security Audit: How Agents Defend Themselves

Security spectrum overview

These agents span an enormous range of security investment. At one end, some agents treat shell access as a trusted operation with minimal guards. At the other, agents implement tree-sitter AST parsing, Zsh attack catalogs, quote-aware pipeline parsers, and binary provenance verification.

Shell injection defense: the full spectrum

1. Claude Code — Tree-sitter AST analysis (23-point validation)

Claude Code's bashSecurity.ts is the deepest shell security implementation in this set. It imports tree-sitter's shell grammar to parse command ASTs before execution — operating on the actual parsed structure, not regex approximations.

The ZSH_DANGEROUS_COMMANDS constant is a JavaScript Set for O(1) lookup. The code comments explicitly state "defense in depth against future changes" — this is security engineering that anticipates novel attack vectors that don't exist yet.

2. Neovate Code — 22-item banned list + quote-aware pipeline parser

Neovate's src/tools/bash.ts uses a character-level state machine to handle quoting correctly before any security check. This matters because naive regex approaches fail on quoted strings:

// State machine tracks: inSingleQuote, inDoubleQuote, escaping
// splitPipelineSegments() respects quoting so 'echo "a|b"' is ONE segment
// hasCommandSubstitution() tracks same states to find $() and backticks

The full banned command list (22 items):

alias, aria2c, axel, bash, chrome, curl, curlie, eval,
firefox, fish, http-prompt, httpie, links, lynx, nc,
rm, safari, sh, source, telnet, w3m, wget, xh, zsh

Beyond static bans, it detects high-risk patterns like rm -rf, sudo, dd if=, mkfs, and curl | sh, checking every pipeline segment individually.

3. Codex — execpolicy rule DSL + safe command allowlist

Codex implements shell security through its execpolicy crate, which provides a rule DSL for fine-grained command control. The policy system supports prefix patterns, network rules, and command whitelists/blacklists. A fast-path is_known_safe_command() allowlist short-circuits common read-only operations like ls, cat, and find without hitting the full policy engine.

Sandboxing varies by platform: macOS uses Seatbelt (sandbox-exec) with a workspace-write profile that keeps .git and .codex read-only; Linux uses bubblewrap (prefers system bwrap, vendored Landlock fallback); Windows uses restricted tokens with split-filesystem policies. The CLI exposes codex sandbox {macos,linux,windows} for testing, plus a --sandbox flag with levels: read-only, workspace-write, and danger-full-access.

The shell backend uses zsh-fork on macOS for safe shell spawning, with classic Unix escalation elsewhere. Permission escalation flows through the request_permissions tool with a runtime approval flow, and MCP tool approvals can be overridden per-tool in config.toml. The codebase enforces strict clippy denies across 70+ crates: unwrap_used, expect_used, needless_borrow, and more.

4. Hermes — Supply-chain verification via Tirith + cosign

Hermes takes a completely different approach: rather than blocking specific commands, it verifies the execution environment. The Tirith security binary is downloaded from GitHub releases with SHA-256 hash verification and optional cosign provenance validation.

This is supply-chain security — verifying that the execution binary is authentic and untampered — not just command blocking. The Tirith binary checks for:

• Homograph URLs (lookalike domains for phishing)
• Pipe-to-interpreter patterns
• Terminal injection attempts

4. Crush — TUI permission gates + SHA-256 loop detection

Crush integrates a permission system directly into its TUI. Before executing dangerous operations, the user sees a permission prompt. This is a UX-level defense — appropriate for an interactive tool where the user is present.

The SHA-256 loop detection in internal/agent/loop_detection.go computes a signature by hashing tool_name + "\x00" + input + "\x00" + output for every tool call. If any signature appears more than 5 times in the last 10 steps, the agent halts.

Prompt injection defense

Prompt injection — tricking the agent via crafted input in files, context, or memory — is a growing attack vector. Only two agents in this set have explicit defense:

Permission systems compared

How agents decide whether to run a command is a fundamental security question. The approaches vary dramatically:

Sandboxing and execution isolation

When things go wrong, how contained is the damage? The agents fall into three camps:

Loop detection as a safety mechanism

Infinite loops aren't just annoying — they can burn through API credits, fill disks with logs, or repeatedly execute destructive commands. Only three agents have explicit loop detection; the rest (including Pi Mono, Claude Code, and Mux) rely on the model to notice repetition or on max-iteration caps.

Security verdicts

What most agents are missing

Several security patterns appear in only one or two agents despite being broadly applicable:

• Prompt injection scanning — Only Hermes does this explicitly. Every other agent (including Pi Mono, Claude Code, and Qwen Code) trusts context files blindly.
• Cross-session memory scanning — Only Hermes scans memory on load to prevent injection propagation.
• Supply-chain verification — Only Hermes verifies binary provenance.
• Tree-sitter AST parsing — Only Claude Code parses shell commands structurally rather than with regex.
• Quote-aware parsing — Only Neovate correctly handles quoting in pipeline segmentation.
• Temperature-adaptive retry — Only OpenHands bumps temperature on deterministic failures.
• Agent-requested compression — Only OpenHands lets the agent ask for context condensation.
• File mutation queues — Only Pi Mono serializes concurrent writes to the same file, preventing a subtle race condition that affects most other agents.
• Diff. TUI rendering — Only Pi Mono redraws only changed terminal cells, reducing attack surface from terminal escape sequence flooding.

Bottom line

The hallmark of a production-ready coding agent is not its prompt engineering, but its deterministic security scaffold. Claude Code and OpenHands represent the bleeding edge of different philosophies: Claude builds walls around the host machine, OpenHands isolates in containers.

Meanwhile, several agents (Pochi, Kimi CLI, Mux) rely primarily on trust and approval workflows rather than active defense mechanisms. This isn't necessarily wrong — for trusted environments, minimal friction is a feature — but it matters enormously when deploying agents on production infrastructure.