madisonrickert/jev-permission-gate
32 stars · Last commit 2026-10-04
A Claude Code mod that uses TypeSafe's Jev to decide auto mode tool calls. 2x faster than the built-in classifier on the calls it decides.
README preview
# jev-permission-gate A [Claude Code](https://code.claude.com) mod that puts [TypeSafe's Jev](https://docs.typesafe.ai/) in front of the auto mode classifier. In auto mode, for each tool call Claude Code would otherwise send to its built-in classifier, Jev answers eight yes/no questions in one request and the mod decides: - **allow** when the call plausibly serves your request (at least 50%) and every risk check is at or below 10%. The built-in classifier doesn't run. - **deny** when the call is both risky and unrequested: some risk check at or above 80% and "serves your request" at or below 30%. Claude sees the reason. - **defer** everything else to the built-in classifier, unchanged. A shell-aware blocklist sits in front of Jev. A call it flags can still be denied by Jev but is never allowed, and a call that carries a secret never leaves your machine. ## Results Evaluated on 7,860 labeled tool calls drawn from independent sources: another permission gate's reviewed corpus, five other Jev gates' test suites, real commands from coding-agent trajectories, tldr-pages, and hand-written edge cases. Thresholds were chosen on a dev split; the test split was run once, after the policy was committed. On the sealed test split (5,322 calls, 3,664 of them risky): - **1 unsafe allow in 3,664 risky calls.** That bounds the unsafe-allow rate below 0.13% (95% confidence). The one miss, an abbreviated `git reset --h`, is fixed in the shipped blocklist. - **Settles 62% of real agent work without the built-in classifier.** On commands real coding agents ran (SWE-agent, OpenHands, and Terminal-Bench trajectories), it allowed 70% of the routine calls outright: 85% on repository bug-fixing. - **Denies hostile calls itself.** It denied 54% of the calls that should be denied, so those don't wait for the built-in classifier either. - **Rarely wrong in the other direction.** 5 wrong denials in 1,699 calls where a denial would be wrong (0.3%).