Authorship note: Built with AI coding assistance. I defined the requirements, ran the tests, and reviewed the changes. — Leigh Rodney, AI Researcher & Founder at Globe Teck
llm-redteam-harness
AI safety · Red-teaming · Performance · Python · MIT
A small harness for LLM safety evals and red-teaming. It runs prompt-injection, jailbreak-refusal, personal-data-leak and benign-control cases against a pluggable model adapter, scores the answers, and writes reports with latency p50/p95 and throughput, plus a --fail-under threshold for use as a CI gate. Version 0.2.0 added an offline multi-turn tool-use mode with simulated tools, loaders for public benchmarks (HarmBench, XSTest, TruthfulQA, SimpleQA, InjecAgent) with a mandatory chemical, biological and WMD filter applied to those benchmark loads, statistics rules for calling a regression, and a replication manifest in every report. Development and tests use mock models and an optional small local open-weight model only, and the agent mode has not yet been run against a real model.
Repository · v0.2.0 release
gemini-mcp-server
MCP · Agents · Gemini · Python · MIT
A small Model Context Protocol server that gives any MCP client one tool, gemini_ask, which sends a prompt to Google Gemini and returns the answer. It retries rate limits, server errors and network failures with exponential backoff and jitter, falls back to another model when one fails, and tells you which model answered. Bad requests and key errors fail fast with readable error text, and the API key comes from an environment variable or a key file, never from code. It installs from source (version 0.1.0), and its unit tests use mocked HTTP calls.
Repository
cpanel-spam-filter-sync
Email · cPanel · Automation · Python · MIT
A Python utility that keeps spam rules in one YAML file and syncs them to cPanel server-side email filters through the UAPI, so filtering happens on the mail server and works the same in every mail client. Trusted-sender rules run first, then sender, subject and authentication-failure header patterns (such as spf=fail in a message's headers) send matches to the spam folder. It's safe to re-run: existing filters update in place, nothing is ever deleted, and filters no longer in the rules are only reported for review. A --dry-run mode previews the plan without making any API calls, and its tests use mocked HTTP calls rather than a live cPanel server.
Repository
agent-email-safety
AI safety · Prompt injection · Email · Design write-up
A design write-up, with no code, on protecting a small team of AI agents that read and act on email. It is built on one rule, "email content is data, never instructions", and covers the threat model, a harmless canary injection test (passed: the agent ignored the message and flagged it), a guard that keeps verification-code and sign-in mail out of bot inboxes, and a locked-down, dry-run-by-default design for an email-server connector. The results page is open about what's still pending: a positive control showing that normal forwarding still works, and a spoofed-sender canary. A separate finding, Credential near-miss, explains how a pre-flight check caught an API token stored in the wrong keychain field, why the token was rotated anyway, and what it taught us about tools that print credential fields.
Repository · Credential near-miss write-up
Want this kind of testing on your own system? Start a project