RFD D18: Plugin Event Subscriptions and Query Delegation

• Status: Draft
• Category: Design
• Authors: Jean Mertz git@jeanmertz.com
• Date: 2026-04-06
• Extends: RFD 072

Summary

This RFD extends the command plugin protocol (RFD 072) with event subscriptions and agent loop delegation. Plugins can subscribe to live conversation events, respond to interactive prompts (tool approval, inquiries), and trigger LLM queries that JP executes on their behalf. Together, these capabilities enable plugins to act as alternative frontends to JP's agent loop — the key building block for a web-based chat interface.

Motivation

RFD 072 defines a request/response protocol that covers workspace queries and mutations. A plugin can list conversations, read events, lock, write, and produce formatted output. This is sufficient for read-only tools (web viewer, exporters) and simple write tools (importers, bulk editors).

It is not sufficient for a chat interface. A chat plugin needs to:

1. Stream LLM responses in real time — tokens must arrive at the plugin as they are generated, not after the full response is complete.
2. Handle tool approval prompts — when the agent loop wants to run a tool, the plugin must present the approval UI and send the decision back.
3. Handle inquiry questions — tools may ask questions that need user input.
4. Observe concurrent activity — if another jp query session writes to the same conversation, the plugin should see those events too.

All four require JP to push messages to the plugin, which the request/response model from RFD 072 does not support. This RFD adds that capability.

Design

Subscriptions

A plugin subscribes to a conversation to receive events as they occur.

Subscribe:

{
  "type": "subscribe",
  "sub_id": "chat-main",
  "conversation": "17127583920",
  "events": [
    "chat_response",
    "tool_call_request",
    "tool_call_response"
  ]
}

Response:

{
  "type": "subscribed",
  "sub_id": "chat-main"
}

The sub_id is chosen by the plugin and must be unique among its active subscriptions. JP echoes it on every pushed event, allowing the plugin to maintain multiple independent subscriptions — e.g., two chat windows side-by-side, or one subscription for rendering and another for logging.

The events filter is optional. When omitted, all event types are delivered.

Once subscribed, JP pushes events as they occur on the conversation (from any source — a concurrent jp query session, another plugin, or this plugin's own query operation):

{
  "type": "event",
  "sub_id": "chat-main",
  "data": {
    "timestamp": "...",
    "type": "chat_response",
    "message": "Here's what I found..."
  }
}

Unsubscribe:

{
  "type": "unsubscribe",
  "sub_id": "chat-main"
}

Response:

{
  "type": "unsubscribed",
  "sub_id": "chat-main"
}

Subscriptions are automatically removed when the plugin exits.

Interactive Events

Some pushed events require a response from the plugin. These carry a respond field indicating what kind of response JP expects.

Tool call approval:

{
  "type": "event",
  "sub_id": "chat-main",
  "respond": "tool_approval",
  "data": {
    "type": "tool_call_request",
    "id": "tc_1",
    "name": "cargo_check",
    "arguments": {}
  }
}

The plugin responds:

{
  "type": "respond",
  "sub_id": "chat-main",
  "respond": "tool_approval",
  "tool_call_id": "tc_1",
  "action": "approve"
}

Valid actions: approve, reject, modify (with an arguments field containing the modified arguments).

Inquiry question:

{
  "type": "event",
  "sub_id": "chat-main",
  "respond": "inquiry",
  "data": {
    "type": "inquiry_request",
    "id": "inq_1",
    "question": "Which file?",
    "options": [
      "a.rs",
      "b.rs"
    ]
  }
}

The plugin responds:

{
  "type": "respond",
  "sub_id": "chat-main",
  "respond": "inquiry",
  "inquiry_id": "inq_1",
  "answer": "a.rs"
}

If the plugin does not respond within a configurable timeout, JP falls back to the default behavior configured for non-interactive mode (auto-approve, reject, etc.).

Events without a respond field are informational — the plugin observes them but does not need to reply.

Query (Agent Loop Delegation)

The query operation triggers JP's agent loop on behalf of the plugin. The plugin sends a user message, JP runs the full turn loop (streaming, tool calls, retries), and events flow back to the plugin via its subscription.

{
  "type": "query",
  "conversation": "17127583920",
  "content": "What files changed?"
}

The query payload supports optional fields for the full range of run_turn_loop inputs:

• attachments (array, optional): Attachments to include in the turn, equivalent to jp query -a. Each entry follows the attachment serialization format (inline content with a type discriminator).
• schema (object, optional): A JSON Schema constraining the assistant's response format, triggering structured output. Equivalent to the schema field on ChatRequest.

Example with both:

{
  "type": "query",
  "conversation": "17127583920",
  "content": "Summarize this file",
  "attachments": [
    {
      "type": "file_content",
      "path": "src/main.rs",
      "content": "fn main() {}"
    }
  ],
  "schema": {
    "type": "object",
    "properties": {
      "summary": {
        "type": "string"
      }
    },
    "required": [
      "summary"
    ]
  }
}

When omitted, attachments defaults to an empty list and schema defaults to null (free-form response).

Response (acknowledges the query has started):

{
  "type": "query_started",
  "conversation": "17127583920"
}

JP locks the conversation, starts a turn, and runs the agent loop. Events stream to the plugin via its active subscription on that conversation. When the turn completes:

{
  "type": "event",
  "sub_id": "chat-main",
  "data": {
    "type": "query_complete"
  }
}

During the turn, the plugin receives all intermediate events: chat_response chunks (for streaming tokens to a browser), tool_call_request events (which may require approval via respond), inquiry_request events, and tool_call_response results. The plugin is effectively an alternative frontend to the same agent loop that powers jp query in the terminal.

The conversation must either be already locked by the plugin or unlocked (in which case JP acquires the lock for the duration of the query and releases it on completion). If another process holds the lock, the request returns an error.

If the plugin sends query without an active subscription on that conversation, JP returns an error — there would be no way to deliver the streaming events or interactive prompts.

Integration with the Agent Loop

The agent loop is currently implemented in jp_cli::cmd::query::turn_loop. Some of its external dependencies are trait-based today:

Dependency	Current abstraction	Status
Prompt backend	PromptBackend trait	Exists
Tool execution	ExecutorSource trait	Exists
Inquiry backend	InquiryBackend trait	Exists
Response output	ChatResponseRenderer	Concrete struct

ChatResponseRenderer is a concrete struct instantiated inside TurnCoordinator::new(). There is no ResponseRenderer trait in the current codebase — TurnCoordinator owns the renderer directly and is itself created inside run_turn_loop. This means a protocol-backed renderer cannot be injected without first extracting rendering behind a trait.

For query delegation, JP provides protocol-backed implementations of the required traits:

• ProtocolPromptBackend: When the turn loop asks for tool approval, this implementation sends a respond: "tool_approval" event to the plugin and blocks until the plugin sends respond back.
• ProtocolResponseRenderer: When the turn loop emits ChatResponse chunks, this implementation pushes them as event messages on the plugin's subscription. Requires the ResponseRenderer trait extraction described above.
• ProtocolInquiryBackend: When a tool asks a question, this implementation sends a respond: "inquiry" event and awaits the answer.

RFD 026 proposes extracting the turn loop into a standalone jp_agent crate with explicit trait boundaries, which would introduce a ResponseRenderer trait as part of that work. If RFD 026 has landed by the time Phase 3 begins, the protocol bridges plug into jp_agent::run_turn_loop() directly. If not, Phase 3 of this RFD must extract TurnCoordinator's rendering into a trait as a prerequisite step before the protocol-backed implementations can be wired in.

Web Chat Example

A web server plugin (jp-serve) with chat support:

1. When a browser opens a conversation, subscribe to it with a unique sub_id per browser session.
2. When the user sends a message, issue a query with the message content.
3. Receive streaming chat_response events via the subscription and forward them to the browser over SSE or WebSocket.
4. Receive tool_call_request events with respond: "tool_approval" and present an approval UI in the browser. Send the user's decision back via respond.
5. Receive query_complete and signal the browser that the turn is done.
6. unsubscribe when the browser disconnects.

The plugin never calls the LLM directly, manages conversation locks, or persists events — JP handles all of that. The plugin is purely a presentation layer.

Drawbacks

• Bidirectional complexity: The protocol shifts from pure request/response to a bidirectional event stream. Plugins must handle unsolicited messages (pushed events) arriving at any time, interleaved with responses to their own requests. Multi-threaded plugins handle this naturally; single-threaded plugins (shell scripts) cannot easily participate in subscriptions.

• Timeout semantics: Interactive events require the plugin to respond within a timeout. The right default timeout and fallback behavior may vary by use case (a web UI might want a long timeout to let the human think; a batch script might want to auto-approve immediately).

• Agent loop coupling: The query operation exposes JP's internal agent loop behavior as a protocol surface. Changes to how the turn loop handles tool calls, retries, or interrupts become visible to plugins. The trait boundaries from RFD 026 help, but the protocol-level contract (which event types arrive and in what order) is an additional compatibility surface.

Alternatives

Plugin calls the LLM directly

The plugin could bypass JP's agent loop entirely: call the LLM API itself, manage tool execution, and use JP only for conversation storage via push_events.

Rejected because:

• Duplicates the agent loop (streaming, retries, tool coordination, inquiry handling) in every chat-capable plugin.
• The plugin would need LLM API keys and provider configuration, breaking the principle that JP manages credentials.
• Tool execution requires access to JP's tool definitions and MCP servers, which are not exposed through the current protocol.

Polling instead of subscriptions

The plugin could repeatedly call read_events to check for new events.

Rejected as the primary mechanism because polling adds latency and wastes resources for streaming use cases. However, polling remains available as a fallback for simple plugins that don't need real-time updates.

Non-Goals

• Multi-conversation queries: A single query operates on one conversation. Orchestrating parallel queries across conversations is the plugin's responsibility.
• Plugin-side tool execution: Tools are executed by JP, not by the plugin. The plugin can approve, reject, or modify tool calls, but cannot provide its own tool implementations through this protocol. (Wasm plugins via RFD 016 serve that purpose.)
• Streaming protocol optimization: The JSON-lines format is kept for consistency with RFD 072. A binary framing format (msgpack, protobuf) is future work if latency becomes measurable.

Risks and Open Questions

• Cross-process event delivery: Subscriptions deliver events from any source. For events generated by the same plugin's query operation, delivery is straightforward (JP controls the turn loop). For events from a concurrent jp query session writing to the same conversation, JP needs a notification mechanism. The current storage layer uses append-only files with flock coordination but has no change notification. Polling the events file periodically is the simplest approach; inotify/kqueue-based notification is an optimization for later.

• Interactive event ordering: When multiple tool calls arrive in a batch, the plugin receives multiple respond: "tool_approval" events. The protocol does not currently define whether the plugin must respond in order or can respond out of order. The simplest rule: responses can arrive in any order, matched by tool_call_id.

• Query cancellation: If the plugin wants to cancel a running query (e.g., the user navigated away in the browser), there is no cancellation message defined. A cancel_query message type would be a natural addition but is deferred until the need is validated.

• Backpressure: If the LLM streams tokens faster than the plugin can consume them (e.g., the browser connection is slow), pushed events queue up in JP's pipe buffer. For human-speed interactions this is unlikely to be a problem. For high-throughput scenarios, a flow control mechanism may be needed.

Implementation Plan

Phase 1: Subscriptions

• Add subscribe, unsubscribe message types to jp_plugin.
• Implement the event push mechanism in JP's dispatcher.
• For events from the same process (e.g., push_events by the same plugin), push to active subscriptions immediately.
• For cross-process events, implement file-based polling as the initial notification mechanism.
• Test with a plugin that watches a conversation while jp query writes to it.
• Depends on RFD 072 Phase 1 (protocol core).

Phase 2: Interactive events

• Add respond field to pushed events and respond message type.
• Implement timeout and fallback behavior.
• Test with a shell script that auto-approves tool calls.
• Depends on Phase 1.

Phase 3: Query delegation

• If RFD 026 has not landed: extract TurnCoordinator's rendering into a ResponseRenderer trait so protocol-backed and terminal-backed renderers can be swapped.
• Implement query, query_started, query_complete message types.
• Build protocol-backed implementations of PromptBackend, ResponseRenderer, and InquiryBackend.
• Support attachments and schema fields in the query payload.
• Bridge into run_turn_loop (via jp_agent if RFD 026 has landed, otherwise via jp_cli internals with the trait extraction above).
• Test with jp-serve serving a chat interface.
• Depends on Phases 1 and 2, and RFD 072 Phase 4 (write operations).

References

• RFD 072: Command Plugin System
• RFD 016: Wasm Plugin Architecture
• RFD 026: Agent Loop Extraction
• RFD 027: Client-Server Query Architecture