Run behavioral eval cases against a live agent.

A case is a YAML file: a prompt sent to the agent as the logged-in member, and deterministic checks run against the reply (contains, regex, verbatim lines from a reference file).

Cases live in the owner's agent repo (e.g.

/evals/*.yaml), versioned next to the behavior they test — and never synced to the agent.

Each case declares the role it is written for (owner or user).

Run owner cases logged in as yourself; for user-role cases either switch to a test-user login (mutiro auth switch, --role user), or probe one of the agent's real user conversations with --in-conversation-with: the probe is sent as you (owner interjection), but the engine answering stays bound to the conversation's own user, so the case exercises that user self — its workspace, its tools, its context.

Examples:

Run the owner-role cases from an agent repo

mutiro agent evals run ./ltl/evals --agent leoltl

Run user-role cases while logged in as a test user

mutiro agent evals run ./ltl/evals --agent leoltl --role user

Run user-role cases against the agent's email self

mutiro agent evals run ./ltl/evals --agent leoltl --role user --in-conversation-with agentmail

Run a single case by name

mutiro agent evals run ./ltl/evals --agent leoltl --only quote-email-template