RFD D23: JSON Input and Schema for CLI

• Status: Draft
• Category: Design
• Authors: Jean Mertz git@jeanmertz.com
• Date: 2026-04-13
• Requires: RFD D19

Summary

This RFD adds a machine-readable JSON interface to JP's CLI. A new --args-schema flag prints a JSON Schema describing the full CLI surface, and a new --args-json flag accepts a JSON object conforming to that schema to drive any JP command. The CLI's existing clap argument structs are the single source of truth — they gain serde::Deserialize and schemars::JsonSchema derives alongside their existing clap derives, so the schema and JSON input/output format fall directly out of the Rust types.

Motivation

JP is a CLI tool designed for interactive use by humans. But two categories of callers need programmatic access:

1. LLMs. An LLM that wants to invoke JP (e.g. as a tool in an agent loop) can be better served with a machine-readable description of JP's interface and a structured way to call it. Today the LLM must construct shell command strings, which is fragile and error-prone.

2. Scripts and automation. CI pipelines, editor integrations, and wrapper tools benefit from JSON input over string-splicing shell arguments.

Without this change, programmatic callers must reverse-engineer the CLI's flag names and value formats from help text, and construct argv arrays by hand. This is both fragile and limits JP's usefulness as a building block.

Design

Guiding Principle

The CLI argument structs (Cli, Globals, Query, Conversation, etc.) are the single source of truth. They already define the interface via clap derives. This RFD adds serde::Deserialize and schemars::JsonSchema to the same structs so that the JSON schema and deserialization behavior are derived from the same types — not maintained as a parallel hierarchy.

Where a field type works naturally with both clap and serde/schemars, no changes are needed beyond adding derives. Where a type is clap-specific (manual FromArgMatches impls, Option<Option<T>> triple-state patterns), the type is replaced or annotated to support all three trait families.

Fields that use custom value_parser functions do not need new bridging types. Clap's value_parser translates CLI string input into the field's target type (e.g. schemars::Schema, DateTime<Utc>, String). Serde deserializes JSON input into the same target type directly. The value_parser functions remain CLI-only conveniences — they are invisible to serde.

User-Facing Interface

Two new global flags provide the two sides of the JSON interface:

Schema output:

jp --args-schema                    # full schema
jp --args-schema query              # schema for `jp query`
jp --args-schema conversation.edit  # schema for `jp conversation edit`

Prints the JSON Schema for the Cli struct (or a subcommand) to stdout and exits. The full schema is generated by calling schemars::schema_for!(Cli).

The optional argument is a dot-delimited command path that prunes the command tree to only the path from root to the target subcommand. Globals, root options, and metadata are always included. At each nesting level, sibling variants are dropped but the structural nesting is preserved — the output is always a valid --args-json schema for invoking that specific subcommand.

For example, jp --args-schema query emits the full top-level schema but with command containing only the query variant (not conversation, config, etc.). jp --args-schema conversation.edit keeps the command.conversation.edit nesting but drops conversation's sibling commands (query, config, ...) and edit's sibling subcommands (ls, show, fork, ...). When omitted, all variants at all levels are included.

Because schemars generates schemas using $ref and definitions, pruning requires walking the generated schema AST, filtering oneOf arrays at each command level, and removing orphaned definitions. This is non-trivial but mechanical.

The output includes JP metadata:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "jp",
  "$jp": {
    "schema_version": 1,
    "jp_version": "0.42.0"
  },
  "type": "object",
  "properties": {
    "...": "..."
  }
}

schema_version is a monotonic integer that bumps only on breaking changes to the JSON input contract. jp_version is informational.

JSON input:

jp --args-json '{ ... }'
echo '{ ... }' | jp --args-json -

Accepts a JSON object conforming to the schema. The JSON is first deserialized into a thin envelope struct that extracts and validates the optional schema_version metadata, then the inner payload is deserialized into the Cli struct and passed into the existing run_inner execution path. The rest of the pipeline (config loading, workspace resolution, command dispatch) is unchanged.

Both flags are handled before clap parsing — they are detected by scanning raw argv. --args-schema prints and exits. --args-json deserializes and dispatches.

Errors are returned as structured JSON when stdout is not a TTY (consistent with existing --format json behavior).

--args-json does not imply non-interactive mode. TTY detection continues to govern interactive behavior as usual. A user invoking --args-json from an interactive terminal gets the same interactive experience (editor prompts, confirmation dialogs) as normal CLI usage.

Stdin interaction. When --args-json - is used, the pre-parse logic consumes stdin to read the JSON payload. This means commands that also read stdin (e.g. Query piping input) will find it empty. JSON callers must embed all input in the JSON payload (e.g. the query field) rather than relying on stdin piping. This should be documented in the CLI help text for --args-json.

Bridging Types

An audit of all ~30 command structs identifies the following categories of field types and the work required for each.

Types that already work

Primitives (bool, String, Option<String>, u8, usize, Option<usize>, Option<NonZeroUsize>), Option<Utf8PathBuf>, and schemars::Schema already have or trivially gain Serialize, Deserialize, and JsonSchema. No changes needed beyond adding derives to the containing struct.

Fields with custom value_parser functions

Four fields use custom value_parser functions to translate CLI string input into typed values: string_or_path (→ String), parse_schema (→ schemars::Schema), parse_fork_turns (→ Option<usize>), and parse_duration (→ DateTime<Utc>).

These do not need bridging types. The target types already implement Deserialize and JsonSchema. Clap uses the value_parser to convert CLI strings; serde deserializes from JSON directly into the target type. The value_parser functions remain CLI-only and are invisible to serde.

For example, Query::schema is Option<schemars::Schema>. On the CLI path, clap applies string_or_path.try_map(parse_schema) to convert the DSL string. On the JSON path, serde deserializes a JSON Schema object natively.

Simple enums

CliFormat, LogFormat, and the two Sort enums (in conversation/ls and conversation/grep) are clap::ValueEnum types. They need Serialize, Deserialize, and JsonSchema added — one derive line each.

ReasoningConfig already has Serialize/Deserialize via schematic's Config derive and needs only JsonSchema.

FromStr types

Seven types (Editor, AttachmentUrlOrPath, WorkspaceIdOrPath, ConversationTarget, KvAssignment, ExpirationDuration, KeyValueOrPath) already implement FromStr. They need Deserialize (delegating to FromStr for string inputs) and JsonSchema (emitting {"type": "string"} with appropriate descriptions or enum values). A shared helper trait or macro can reduce boilerplate across these.

Manual clap types

Three types have hand-written clap::Args / FromArgMatches impls:

• ToolDirectives: An ordered list of tool enable/disable operations. Needs manual Deserialize and JsonSchema impls representing the same concept as a JSON array of directive objects. Because these types represent lists (not maps), they must not use #[serde(flatten)] — they are regular named fields in the serde representation.

• FlagIds<S, M> and PositionalIds<S, M>: Lists of conversation targets. For JSON, both collapse to Vec<ConversationTarget> — the positional-vs-flag distinction is a CLI-only concern. Need manual Deserialize and JsonSchema impls. Same caveat about not using #[serde(flatten)].

Triple-state fields (Option<Option<T>>)

Nine fields across Query, Edit, Fork, and Print use clap's num_args = 0..=1 with default_missing_value to represent three states: absent, flag-present-without-value, flag-present-with-value.

For the JSON path, the three states map to: field absent or undefined, null or true, and a concrete value. These fields need custom serde handling to distinguish absent from null.

The exact mechanism needs prototyping. Options include a TriState<T> newtype that works as a clap field type, per-field #[serde(default, deserialize_with)] annotations, or a combination. Clap's derive macro has special handling for Option<Option<T>> tied to num_args, so a replacement type must either replicate that behavior or keep Option<Option<T>> for clap and use serde attributes for the JSON path. This will be resolved during Phase 1 implementation.

Commands::External

The #[command(external_subcommand)] variant currently holds Vec<String>. Clap's external_subcommand requires a type implementing FromIterator<String>. For JSON, the raw vector is not ergonomic — a structured object with name and args fields is better.

The variant becomes External(ExternalCommand), where ExternalCommand is a struct that implements FromIterator<String> (for clap), Serialize, Deserialize, and JsonSchema (for the JSON path).

For --args-schema, installed plugins that implement RFD D19's Help protocol contribute their structured arg definitions to the schema as oneOf variants under the External discriminator. Plugins that don't implement Help get an opaque {"name": "string", "args": ["string"]} fallback.

RFD D19's PluginArg type needs a small extension for this: an optional value_type field ("string" | "integer" | "number" | "boolean", defaulting to "string") so the schema can emit proper JSON Schema types for plugin arguments.

Schema Versioning

The schema version ($jp.schema_version) tracks breaking changes to the JSON input contract — field renames, removed fields, structural changes to command nesting. It is independent of JP's release version.

Adding new optional fields is not a breaking change and does not bump the schema version. Removing or renaming fields is breaking. Changing a field's type is breaking.

The schema_version field in the input JSON is optional. When present, JP validates it matches the expected version and returns an error on mismatch. Validation happens in the envelope struct before the inner payload is deserialized into Cli.

Drawbacks

Annotation overhead. Every command struct gains three additional derives and some fields need #[serde(...)] / #[schemars(...)] attributes. This adds visual noise to the arg definitions. In practice, most structs need only the derive line — the handful of fields needing attributes are concentrated in Query and the conversation targeting types.

Dual maintenance of manual impls. The three types with manual clap::Args impls (ToolDirectives, FlagIds, PositionalIds) now also need manual Deserialize and JsonSchema impls. These must be kept in sync with the clap behavior. Tests that round-trip CLI args through JSON can catch drift.

clap constraint gaps. Clap's conflicts_with and requires constraints are not automatically enforced on the serde deserialization path. Invalid combinations (e.g. both edit and no_edit set to true) are caught by the execution pipeline rather than at parse time. This is acceptable for the initial implementation; a validate() pass can be added incrementally.

Binary size. schemars adds code generation for schema derivation. The impact is expected to be modest since schemars is already a dependency (used by Query::schema), but should be measured.

Alternatives

Generate schema from clap introspection at runtime

Walk clap::Command at runtime using its introspection API (get_subcommands, get_arguments, etc.) to produce a JSON Schema.

Rejected because clap's introspection is lossy — value parsers are opaque, so the schema would emit "string" for nearly every argument. The core motivation for the schema is to give LLMs precise type information, which requires the richer type data that schemars::JsonSchema provides.

Dedicated parallel schema structs

Maintain a separate set of structs annotated only with schemars that mirror the CLI shape, with a conversion layer between them.

Rejected because two sources of truth means every CLI change requires a corresponding schema struct update. This maintenance burden grows with every new command and flag.

JSON → synthetic argv → clap parsing

For --args-json, convert the JSON object into a Vec<String> argv and feed it through clap's normal parsing path.

This is simpler to implement but lossy in the other direction — some JSON structures (nested objects for --cfg, ordered arrays for ToolDirectives) have no natural argv representation. It also means every JSON input is subject to clap's string-based parsing quirks. Direct deserialization into the struct is more robust and preserves types end-to-end.

Non-Goals

• Full clap constraint enforcement in JSON mode. Constraints like conflicts_with and requires are not replicated in the Deserialize path. The execution pipeline catches real conflicts at runtime.

• Shell completions from the schema. The JSON Schema could in theory generate shell completions, but that is out of scope. See RFD 059.

• Non-interactive mode. --args-json does not imply non-interactive behavior. See RFD 049 for the --non-interactive flag.

• --cfg JSON objects. Accepting JSON objects as --cfg values is a useful extension to the config pipeline but orthogonal to the JSON CLI interface. It is tracked separately.

• Long-running JSON API server. The schema defined here could serve as the request format for a persistent JSON-over-HTTP (or JSON-over-stdio) server that accepts JP commands without process-per-invocation overhead. That is a separate design concern, but this RFD's schema is intended to be reusable as the request contract for such a server.

Risks and Open Questions

1. schemars compatibility with schematic types. Several CLI fields use types from jp_config that are derived via schematic's Config macro. These types have Serialize/Deserialize but may not have JsonSchema. Each needs to be checked — adding JsonSchema to schematic-derived types may require #[schemars(with = "...")] annotations or wrapper types.

2. Schema stability. Once external consumers depend on the schema, breaking changes become costly. The schema_version mechanism helps, but the initial schema should be designed carefully. Starting with Query as the first fully-schema'd command (the primary use case for LLM callers) allows iteration before committing to the full surface area.

3. schemars version. The project uses schemars 1.0.0-alpha.17. If the alpha API changes before 1.0 stable, derives and custom impls may need updating.

4. Enum representation for Commands. Serde's default externally-tagged enum produces {"variant_name": {fields}}. This is clean but the exact tag name and nesting for subcommands (e.g. {"conversation": {"ls": {fields}}}) needs to be validated against what schemars generates and whether LLMs handle nested discriminators well.

5. Negative flags. CLI has --no-edit, --no-reasoning, --no-tool-use, etc. In JSON, these are better expressed as the positive field set to false (e.g. "edit": false instead of "no_edit": true). The serde representation needs to decide whether to mirror the CLI naming or optimize for JSON ergonomics. Mirroring CLI naming is the default (since the structs are the source of truth), but #[serde(rename)] can alias where it matters.

6. Triple-state clap interaction. Clap's Option<Option<T>> has special derive macro support that a replacement type may not receive automatically. The exact bridging strategy (new type vs serde attributes vs hybrid) needs prototyping.

Implementation Plan

Phase 1: Bridging Types

Create the types and annotations that command structs depend on:

• Triple-state handling for Option<Option<T>> fields (prototyping needed to determine: TriState<T> newtype, per-field #[serde(deserialize_with)], or hybrid approach)
• ExternalCommand struct with FromIterator<String>, Serialize, Deserialize, JsonSchema
• Deserialize + JsonSchema impls for FromStr-based types (ConversationTarget, AttachmentUrlOrPath, WorkspaceIdOrPath, Editor, KvAssignment, ExpirationDuration)
• Deserialize + JsonSchema impls for manual-clap types (ToolDirectives, FlagIds, PositionalIds)

Can be reviewed and merged independently. No user-visible behavior change.

Phase 2: Derive Additions — Query and Globals

Add Serialize, Deserialize, JsonSchema to Cli, Globals, RootOpts, Commands, and Query. This covers the primary use case (LLM callers invoking jp query).

Validate that schemars::schema_for!(Cli) produces a usable schema. Write round-trip tests: construct a Query from JSON, verify it produces the same PartialAppConfig as the equivalent CLI args.

Phase 3: --args-schema and --args-json

Add the pre-parse detection for both flags. --args-schema generates and prints the schema with embedded schema_version and jp_version metadata, with optional command-path pruning of the schema AST. --args-json deserializes the input via the envelope struct, validates schema version if present, and dispatches into run_inner.

These two flags are shipped together — the schema defines the contract, the input consumes it.

Phase 4: Remaining Commands

Add derives to the remaining command structs incrementally: Conversation (and all subcommands), Config, Attachment, Plugin, Init.

Each subcommand tree can be a separate PR.

Phase 5: External Plugin Schema (depends on RFD D19)

Extend PluginArg with value_type. On --args-schema, query installed plugins via the Help protocol and inject their schemas into the External variant.

References

• RFD 049: Non-Interactive Mode and Detached Prompt Policy
• RFD 059: Shell Completions and Man Pages
• RFD 072: Command Plugin System
• RFD D19: Structured Plugin Help Protocol