Services

We help teams measure how their AI systems behave, both how safe they are and how they perform, before users find the problems. Every engagement starts with a written scope and ends with deliverables you keep. Pricing for every service: contact for a quote.

Engagement models:

Start here: most clients begin with one of two offers, the LLM Red-Team & Performance Eval Setup or the AI Agent Safety Audit. Both are fixed scope, and either can lead into Monthly Eval Support.

1. LLM Red-Team & Performance Eval Setup lead offer

We connect our MIT-licensed llm-redteam-harness to your OpenAI-compatible endpoint and build a test suite around your product.

  • A private suite of prompt-injection, jailbreak-refusal, personal-data-leak and benign-control cases. Benign controls catch over-refusal, so safety doesn't break normal use.
  • Performance numbers from the same run: latency p50/p95, throughput, retries and timeouts at your chosen concurrency.
  • A --fail-under threshold so the suite can act as a pass/fail gate in CI.
  • Reports in JSON and Markdown, with a replication manifest (model, dataset hashes, seeds, settings) so a run can be repeated.
  • Optional repeated runs with the harness's statistics rules for calling a regression, instead of reacting to one noisy result.

Deliverables: configured harness, private test suite, CI gate settings and a first written report.

What to know: the standard run is single-turn. The harness's multi-turn tool-use mode works offline with simulated tools and has so far been tested only with a mock model, so agent testing against a live model is scoped per engagement. Refusal and leak checks are pattern-based, which is fast and transparent but can miss paraphrases. An optional judge model can add nuance and brings its own biases. No test suite proves a system is safe. We show measurable risk, not a guarantee.

2. AI Agent Safety Audit fixed scope

A checklist-driven review of how your agent is wired, not just what the model says. It draws on the approach in our agent-email-safety write-up.

  • Untrusted input: can text in an email, web page or document make the agent act? The rule we test for: content is data, never instructions.
  • Tool and connector permissions: least privilege, allow-lists, dry-run defaults and no destructive tools the job doesn't need.
  • Secrets handling: where keys and tokens are stored, whether any tool or log prints them, and how fast they can be rotated.
  • Human sign-off: which actions the agent may take alone and which need a person to approve them.
  • Canary tests: harmless test messages that check whether the agent ignores and flags injected instructions.

Deliverables: findings checklist, risk-ranked fixes and a short readout call.

3. Monthly Eval Support retainer

  • Scheduled regression runs after model, prompt or tool changes
  • Suite upkeep: new cases as your product changes, plus cases for any issue we find
  • Report reviews and a set number of support hours each month

4. Evaluation data and eval-set design by request

The "AI training" side of our work: well-built evaluation and feedback data for the agents and models you're training or tuning.

  • Test cases with clear pass/fail criteria, plus benign controls
  • Scorecards you can track across releases
  • Written test plans and acceptance criteria, using the same discipline as a classic performance test plan

5. Performance testing for LLM endpoints by request

  • Controlled-concurrency runs against your endpoint, reporting p50/p95 latency, throughput, error and retry rates
  • Fair side-by-side comparisons of models, prompts or providers before you switch
  • Cost per task, where needed, is estimated from your provider's usage data. The harness itself doesn't measure cost.

6. Gemini second-opinion integration by request

  • We set up our gemini-mcp-server so your MCP-capable agent can ask Google Gemini for a cross-check or review.
  • Runs on your own Gemini API key. We don't resell API access, and Google's terms apply to your use.
  • Model choice and fallback order are set for your project. We pin specific model versions where you need repeatable results.
  • It installs from source. Its tests use mocked HTTP calls, so we confirm it against your account during setup.

7. Agent setup sprint and workshops by request

  • Agent setup sprint: assistants, connectors and multi-agent workflows for a small team, with approval rules and usage limits set from day one.
  • Workshops: private sessions on red-teaming your own agents, eval design and guardrails for agent teams. Format and dates are set per request.

Ground rules

  • We test only systems you own or are authorized to test.
  • You provide your own API keys and accept your model provider's terms.
  • Hosted models are tested only through official APIs and within the provider's terms.
  • Your test cases, prompts and reports stay private to your engagement and are never added to our public repos.
  • Our public tools stay under their MIT licenses. Your engagement deliverables are yours once paid in full.
  • Our work uses AI-assisted tools, and a person reviews every deliverable.

Have an AI system you need to trust?

Tell us what you're building and we'll scope a test plan. Pricing: contact for a quote.

Start a project