Brain Contract (Chatbridge Protocol)

Chatbridge Protocol

Purpose

This document is the canonical machine-facing contract for implementing a Mutiro chat brain over mutiro agent host --mode=bridge.

If you are deciding whether to swap the brain at all, start with Swap the Built-in Brain — it has a minimal working echo brain. This page is the full protocol reference for building a real one. Reference implementations are listed at the end.

Audience:

coding agents
external brain implementers
first-party integrations that must obey the portable host↔brain boundary

Non-goals:

owner/admin operations
presence control
memory APIs
working memory APIs
scheduler APIs

The bridge is chat-lane only.

Process Topology

Your brain is the parent process. It spawns mutiro agent host --mode=bridge as a child and speaks NDJSON over the child's stdio. The host emits ready first; your brain then drives session.initialize and subscription.set, and afterwards answers host → brain requests such as message.observed.

Inbound message normalization, transcription, attachment preprocessing, and observed-turn shaping are host-owned — your brain receives clean, digested envelopes.

Transport

transport: NDJSON over stdio
protocol version: mutiro.agent.bridge.v1
stdout: protocol only
stderr: diagnostics only

Each line is one complete envelope.

Envelope Schema

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "<envelope_type>", "timestamp": "2026-04-08T12:34:56Z", "request_id": "optional-but-required-for-request-response", "conversation_id": "optional", "message_id": "optional", "reply_to_message_id": "optional", "payload": { "@type": "type.googleapis.com/<protobuf.message>", "...": "..." }, "error": { "code": "optional on type=error", "message": "optional on type=error", "retryable": false } }

Rules:

every request expecting a direct response MUST include request_id
direct responses use the same request_id
success response type is command_result
failure response type is error
conversation_id, message_id, and reply_to_message_id are envelope routing hints; payload fields remain authoritative when present
exception: on turn.end the envelope fields are load-bearing (see its entry), and on message.send an envelope conversation_id back-fills the payload — so direct to_username sends must omit it

Auth And Identity

auth model is agent-key-only
the host is booted with the agent API key
the brain never receives or sends owner JWTs
the brain never receives or sends owner usernames as auth material
the brain cannot impersonate another member
all outbound operations execute as the booted agent identity
media upload is host-owned; the brain never talks to storage directly

Capability Model

Host `ready`

Host role is host.

Host core capabilities:

session.initialize
subscription.set
conversation.list
conversation.get
message.send
message.forward
message.react
turn.end

Host optional capabilities:

message.send_voice
signal.emit
media.upload
recall.search
recall.get

Optional capabilities are advertised only when the host has backing services connected.

Brain capabilities

What your brain must handle when the host sends it:

message.observed (required)
session.observed (required)
task.request (required)
session.snapshot (optional)

Your brain does not announce these in the handshake — the host sends them and expects command_result or error back.

Startup Handshake

Command:

mutiro agent host --mode=bridge --api-key "$MUTIRO_AGENT_API_KEY"

Sequence:

host starts
host emits ready
brain sends session.initialize
host returns command_result
brain sends subscription.set
host begins streaming host→brain bridge requests such as message.observed

The host allows about 10 seconds for steps 3–5: complete subscription.set promptly after launch or the host exits.

Wire discipline

decoding is strict: an unknown or extra field in any envelope is an error
one malformed line (invalid JSON or invalid envelope) terminates the transport — all pending requests fail; there is no per-line recovery
lines are capped at 1 MiB
brain → host commands are processed one at a time, in order; a slow command (such as media.upload) queues the commands behind it. Replies to host → brain requests are matched separately and are not blocked.

`session.initialize`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeInitializeCommand
purpose: open the bridge session; must precede subscription.set

`subscription.set`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeSubscriptionSetCommand
purpose: declare which conversations the brain wants delivered

Rules:

set all: true, or provide at least one conversation_ids entry
rejected before session.initialize
replay fields (replay_recent, from_hashes) are accepted but ignored in bridge mode — only the all/ids subscription is stored

Message Catalog

Host → Brain

`message.observed`

direction: host → brain
payload type: mutiro.chatbridge.ChatBridgeMessageObserved
purpose: deliver one normalized inbound conversation message

Required payload fields:

message.id
message.conversation_id
message.from.username

Notes:

this is the normal inbound turn entry point
attachment preprocessing happens host-side before delivery

Message parts

The message.parts array contains normalized ChatBridgeMessagePart objects. Each part has a type string discriminator and type-specific fields. The host digests the raw wire format into this clean shape before delivery, so brain implementations only need to care about the fields documented below.

type	when it arrives	key fields
`text`	User sends a normal typed message	`text`
`audio`	User sends a voice message (host transcribes upstream)	`transcript`, `url`, `mime_type`, `metadata.duration_ms`
`image`	User shares a photo or screenshot	`url`, `mime_type`, `metadata.caption`, `metadata.width`, `metadata.height`
`file`	User shares a document (PDF, etc.)	`url`, `filename`, `mime_type`, `metadata.caption`
`card`	An agent sends an interactive A2UI card into the chat	`a2ui_json`, `schema_version`, `card_id`
`card_action`	A user clicks or submits on an interactive card	`card_id`, `card_message_id`, `action_id`, `data_json`
`contact`	User shares another member's contact	`metadata.contact_username`, `metadata.contact_display_name`, `metadata.contact_member_type`
`reaction`	User adds or removes an emoji reaction on a message	`reaction` (emoji), `reaction_operation` (`added` or `removed`)
`live_call`	A voice call ends — system posts summary to the thread	`call_id`, `summary_text`, `duration_ms`, `end_reason`, `action_items`, `follow_ups`

Every part also carries an id field (e.g. part-0, part-1). The payload can additionally carry inline images, a reply_to_message_preview, and note parts — treat unknown part types as opaque and skip them.

Attachment downloads and `attachment_context`

For image and file parts, the host downloads the actual files from storage into {agent_workspace}/Downloads/ before delivering the envelope. The payload includes an attachment_context string with a system notification listing the downloaded files and their local paths:

[SYSTEM: Downloaded 2 attachment(s) to your workspace: • photo.jpg → /workspace/Downloads/photo.jpg Image: 1920x1080 pixels, JPG, 2.1 MB • report.pdf → /workspace/Downloads/report.pdf File type: PDF, 450.3 KB These files are now available in your workspace. You can read, analyze, or reference them as needed.]

This means the brain receives both:

The structured part metadata (type, caption, filename, dimensions, etc.) in message.parts
The concrete local file paths in attachment_context, so the brain can read or analyze the files directly from its workspace

The brain should append attachment_context to the message text so the LLM sees both the part descriptions and the file paths.

Example:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.observed", "request_id": "host-17", "conversation_id": "conv_123", "message_id": "msg_abc", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeMessageObserved", "message": { "id": "msg_abc", "conversation_id": "conv_123", "from": { "username": "alex" }, "text": "Can you send the latest sketch?", "metadata": { "client.locale": "en-NZ" } } } }

Success response:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "command_result", "request_id": "host-17", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeCommandResult", "ok": true, "response": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeMessageObservedResult" } } }

`session.observed`

direction: host → brain
payload type: mutiro.chatbridge.ChatBridgeSessionObserved
purpose: deliver non-user-authored session context

`task.request`

direction: host → brain
payload type: mutiro.chatbridge.ChatBridgeTaskRequest
purpose: delegated live/chat work

Required result payload:

mutiro.chatbridge.ChatBridgeTaskResult
text must contain the plain-text delegated result

This is the one required live-lane hook.

`session.snapshot`

direction: host → brain
payload type: mutiro.chatbridge.ChatBridgeSessionSnapshotRequest
purpose: best-effort live bootstrap snapshot

Portable contract:

snapshot shape is brain-defined
no memory-specific fields are required by the portable contract

Recommended fields for good live interop:

system_instruction
recent_messages
tool_hints
metadata

Brain → Host

`message.send`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeSendMessageCommand
purpose: canonical portable send operation for authored outbound messages

Destination rules:

provide exactly one of:
- conversation_id
- to_username
use conversation_id for normal in-thread replies
use to_username for direct user-targeted sends — and omit the envelope conversation_id on those sends (an envelope conversation id back-fills the payload and then fails the exactly-one rule)

Content rules:

provide exactly one of:
- text
- parts

Host translation:

conversation_id maps to backend SendToConversation
to_username maps to backend SendMessage

Supported content:

text
parts

Supported parts are whatever the backend mutiro.messaging.MessagePart supports, including:

text
audio
image
file
card
card_interaction
live_call
reaction
contact

Replying is expressed by reply_to_message_id.

Example, plain text:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.send", "request_id": "brain-4", "conversation_id": "conv_123", "reply_to_message_id": "msg_abc", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeSendMessageCommand", "conversation_id": "conv_123", "reply_to_message_id": "msg_abc", "text": { "text": "Uploading it now." } } }

Example, direct send:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.send", "request_id": "brain-5", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeSendMessageCommand", "to_username": "alex", "text": { "text": "Trying the direct-send path." } } }

Example, parts:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.send", "request_id": "brain-6", "conversation_id": "conv_123", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeSendMessageCommand", "conversation_id": "conv_123", "parts": { "parts": [ { "text": { "text": "Here is the sketch." } }, { "image": { "url": "gs://bucket/path/sketch.png", "caption": "Latest revision", "mime_type": "image/png", "size_bytes": "48122" } } ] } } }

Response type:

payload type: mutiro.messaging.SendToConversationResponse
host normalizes backend SendMessageResponse into this same response shape for username-targeted sends

`message.send_voice`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeSendVoiceMessageCommand
purpose: send a host-synthesized voice message while keeping TTS privileged on the host side

Destination rules:

provide exactly one of:
- conversation_id
- to_username

Content rules:

speech is required
voice_name is optional
reply_to_message_id is optional but should be included for in-thread replies

Host behavior:

the brain does not call audio services directly
the host performs TTS using its privileged audio client
the host sends the resulting audio part through messaging
if the host has no audio client, it must not advertise message.send_voice

Example in-thread voice reply:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.send_voice", "request_id": "brain-voice-1", "conversation_id": "conv_123", "reply_to_message_id": "msg_user_1", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeSendVoiceMessageCommand", "conversation_id": "conv_123", "reply_to_message_id": "msg_user_1", "speech": "I recorded a quick update for you.", "voice_name": "en-US-Chirp3-HD-Orus" } }

Example direct voice send by username:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.send_voice", "request_id": "brain-voice-dm-1", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeSendVoiceMessageCommand", "to_username": "alex", "speech": "Quick voice note from the bridge.", "voice_name": "pt-BR-Chirp3-HD-Orus" } }

`message.forward`

direction: brain → host
payload type: mutiro.messaging.ForwardMessageRequest
purpose: backend-native forward

Rules:

use destination conversation_id
do not emulate forward by copy in the brain

Example:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.forward", "request_id": "brain-6", "conversation_id": "conv_target", "message_id": "msg_source", "payload": { "@type": "type.googleapis.com/mutiro.messaging.ForwardMessageRequest", "message_id": "msg_source", "conversation_id": "conv_target", "comment": "Forwarding the design reference." } }

Response type:

payload type: mutiro.messaging.ForwardMessageResponse

`message.react`

direction: brain → host
payload type: mutiro.messaging.AddReactionRequest
purpose: react to an existing message

Example:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "message.react", "request_id": "brain-7", "message_id": "msg_abc", "payload": { "@type": "type.googleapis.com/mutiro.messaging.AddReactionRequest", "message_id": "msg_abc", "emoji": "👍" } }

Response type:

payload type: mutiro.messaging.AddReactionResponse

`media.upload`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeMediaUploadCommand
purpose: upload local workspace media through the host

Rules:

v1 is path-first
local_path is required
the brain does not upload bytes directly to storage

Example upload request:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "media.upload", "request_id": "brain-8", "conversation_id": "conv_123", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeMediaUploadCommand", "local_path": "/workspace/Downloads/sketch.png", "filename": "sketch.png", "mime_type": "image/png" } }

Example upload response:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "command_result", "request_id": "brain-8", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeCommandResult", "ok": true, "response": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeMediaUploadResult", "media": { "url": "gs://bucket/conversations/conv_123/image/sketch.png", "mime_type": "image/png", "filename": "sketch.png", "size_bytes": "48122", "file_type": "image" } } } }

Then use the returned media reference inside message.send.parts.image|audio|file.

`signal.emit`

direction: brain → host
payload type: mutiro.signal.SendSignalRequest
purpose: emit ephemeral conversation-scoped agent signals
capability: optional

Bound:

known stable signal semantics come from backend enum mutiro.signal.SignalType
unknown semantics use SIGNAL_TYPE_CUSTOM
detail_text is required for SIGNAL_TYPE_CUSTOM

Example:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "signal.emit", "request_id": "brain-9", "conversation_id": "conv_123", "reply_to_message_id": "msg_abc", "payload": { "@type": "type.googleapis.com/mutiro.signal.SendSignalRequest", "conversation_id": "conv_123", "signal_type": "SIGNAL_TYPE_CREATING_IMAGE", "detail_text": "", "in_reply_to": "msg_abc" } }

`turn.end`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeTurnEndCommand
purpose: explicit end-of-turn marker

Rules:

the envelope conversation_id is required
set the envelope reply_to_message_id to the message that started the turn — it settles that message's cursor and read receipt; omitting it leaves them pending

Example:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "turn.end", "request_id": "brain-10", "conversation_id": "conv_123", "reply_to_message_id": "msg_abc", "payload": { "@type": "type.googleapis.com/mutiro.chatbridge.ChatBridgeTurnEndCommand", "status": "completed", "reason": "" } }

`recall.search`

direction: brain → host
payload type: mutiro.recall.RecallSearchRequest
capability: optional

`recall.get`

direction: brain → host
payload type: mutiro.recall.RecallGetRequest
capability: optional

`conversation.list`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeListConversationsCommand
purpose: list visible conversations

`conversation.get`

direction: brain → host
payload type: mutiro.chatbridge.ChatBridgeGetConversationCommand
purpose: get one conversation by:
- conversation_id
- with_member_username

Use with_member_username when the brain needs conversation metadata for a direct chat. It is not required for ordinary direct sends because message.send supports to_username.

`host.shutdown`

direction: brain → host
payload: none
purpose: request a graceful host shutdown

Rules:

the host acks with command_result for the same request_id, then stops the bridge
after the ack, expect no further envelopes

Host Event Stream

The bridge-mode host emits ready once at startup, then only command_result, error, and host → brain requests such as message.observed. Message delivery is message.observed — do not wait on an event stream.

The envelope type space also reserves status, event.message, event.conversation_update, event.message_status, event.signal, event.sync_started, and event.sync_complete for host event streaming; the bridge-mode host does not emit them today.

Error Model

Error envelope:

{ "protocol_version": "mutiro.agent.bridge.v1", "type": "error", "request_id": "brain-11", "conversation_id": "conv_123", "error": { "code": "command_failed", "message": "message.send requires conversation_id or to_username", "retryable": false } }

Codes the bridge-mode host emits:

invalid_json — the line was not valid JSON (fatal: the transport terminates)
invalid_envelope — the envelope failed strict decoding (fatal: the transport terminates)
command_failed — everything else: validation failures and backend errors both surface here, with the detail in message

The type space also reserves invalid_argument, unsupported_envelope, not_found, permission_denied, and unavailable; do not depend on receiving them today.

Retry guidance:

retryable is currently never set true; do not blindly replay non-idempotent mutations on failure — parse message and decide

Out Of Scope

Do not expect the bridge to provide:

memory search/get/write
working memory get/update
scheduler create/list/cancel
owner/admin controls
presence controls
read receipts
direct storage credentials

Live Interop

Live is not part of chatbridge.

To interoperate well with live mode, a bridge brain must implement:

task.request

And should implement:

session.snapshot

task.request result must return:

mutiro.chatbridge.ChatBridgeTaskResult
field: text

That text should be concise plain text suitable for later relay in the live lane.

Reference Implementations

Open-source brains maintained over this same contract — drop-in replacements and templates for building your own:

pi-brain — small and pedagogical, the closest thing to "minimum viable brain over stdio"; read this first when learning the protocol
openclaw-brain — external brain implementation over the bridge contract
claude-agent-brain — wires a coding-agent runtime into a Mutiro agent over the same bridge

The tutorial with a minimal echo brain is Swap the Built-in Brain.

Chatbridge Protocol

Purpose

Process Topology

Transport

Envelope Schema

Auth And Identity

Capability Model

Host ready

Brain capabilities

Startup Handshake

Wire discipline

session.initialize

subscription.set

Message Catalog

Host → Brain

message.observed

Message parts

Attachment downloads and attachment_context

session.observed

task.request

session.snapshot

Brain → Host

message.send

message.send_voice

message.forward

message.react

media.upload

signal.emit

turn.end

recall.search

recall.get

conversation.list

conversation.get

host.shutdown

Host Event Stream

Error Model

Out Of Scope

Live Interop

Reference Implementations

Host `ready`

`session.initialize`

`subscription.set`

`message.observed`

Attachment downloads and `attachment_context`

`session.observed`

`task.request`

`session.snapshot`

`message.send`

`message.send_voice`

`message.forward`

`message.react`

`media.upload`

`signal.emit`

`turn.end`

`recall.search`

`recall.get`

`conversation.list`

`conversation.get`

`host.shutdown`