# RFD D23: JSON Input and Schema for CLI - **Status**: Draft - **Category**: Design - **Authors**: Jean Mertz - **Date**: 2026-04-13 - **Requires**: [RFD D19] ## Summary This RFD adds a machine-readable JSON interface to JP's CLI. A new `--args-schema` flag prints a JSON Schema describing the full CLI surface, and a new `--args-json` flag accepts a JSON object conforming to that schema to drive any JP command. The CLI's existing clap argument structs are the single source of truth — they gain `serde::Deserialize` and `schemars::JsonSchema` derives alongside their existing `clap` derives, so the schema and JSON input/output format fall directly out of the Rust types. ## Motivation JP is a CLI tool designed for interactive use by humans. But two categories of callers need programmatic access: 1. **LLMs.** An LLM that wants to invoke JP (e.g. as a tool in an agent loop) can be better served with a machine-readable description of JP's interface and a structured way to call it. Today the LLM must construct shell command strings, which is fragile and error-prone. 2. **Scripts and automation.** CI pipelines, editor integrations, and wrapper tools benefit from JSON input over string-splicing shell arguments. Without this change, programmatic callers must reverse-engineer the CLI's flag names and value formats from help text, and construct argv arrays by hand. This is both fragile and limits JP's usefulness as a building block. ## Design ### Guiding Principle The CLI argument structs (`Cli`, `Globals`, `Query`, `Conversation`, etc.) are the single source of truth. They already define the interface via `clap` derives. This RFD adds `serde::Deserialize` and `schemars::JsonSchema` to the same structs so that the JSON schema and deserialization behavior are derived from the same types — not maintained as a parallel hierarchy. Where a field type works naturally with both clap and serde/schemars, no changes are needed beyond adding derives. Where a type is clap-specific (manual `FromArgMatches` impls, `Option>` triple-state patterns), the type is replaced or annotated to support all three trait families. Fields that use custom `value_parser` functions do not need new bridging types. Clap's `value_parser` translates CLI string input into the field's target type (e.g. `schemars::Schema`, `DateTime`, `String`). Serde deserializes JSON input into the same target type directly. The `value_parser` functions remain CLI-only conveniences — they are invisible to serde. ### User-Facing Interface Two new global flags provide the two sides of the JSON interface: **Schema output:** ```shell jp --args-schema # full schema jp --args-schema query # schema for `jp query` jp --args-schema conversation.edit # schema for `jp conversation edit` ``` Prints the JSON Schema for the `Cli` struct (or a subcommand) to stdout and exits. The full schema is generated by calling `schemars::schema_for!(Cli)`. The optional argument is a dot-delimited command path that prunes the command tree to only the path from root to the target subcommand. Globals, root options, and metadata are always included. At each nesting level, sibling variants are dropped but the structural nesting is preserved — the output is always a valid `--args-json` schema for invoking that specific subcommand. For example, `jp --args-schema query` emits the full top-level schema but with `command` containing only the `query` variant (not `conversation`, `config`, etc.). `jp --args-schema conversation.edit` keeps the `command.conversation.edit` nesting but drops `conversation`'s sibling commands (`query`, `config`, ...) and `edit`'s sibling subcommands (`ls`, `show`, `fork`, ...). When omitted, all variants at all levels are included. Because `schemars` generates schemas using `$ref` and `definitions`, pruning requires walking the generated schema AST, filtering `oneOf` arrays at each command level, and removing orphaned definitions. This is non-trivial but mechanical. The output includes JP metadata: ```json { "$schema": "https://json-schema.org/draft/2020-12/schema", "title": "jp", "$jp": { "schema_version": 1, "jp_version": "0.42.0" }, "type": "object", "properties": { "...": "..." } } ``` `schema_version` is a monotonic integer that bumps only on breaking changes to the JSON input contract. `jp_version` is informational. **JSON input:** ```shell jp --args-json '{ ... }' echo '{ ... }' | jp --args-json - ``` Accepts a JSON object conforming to the schema. The JSON is first deserialized into a thin envelope struct that extracts and validates the optional `schema_version` metadata, then the inner payload is deserialized into the `Cli` struct and passed into the existing `run_inner` execution path. The rest of the pipeline (config loading, workspace resolution, command dispatch) is unchanged. Both flags are handled before clap parsing — they are detected by scanning raw `argv`. `--args-schema` prints and exits. `--args-json` deserializes and dispatches. Errors are returned as structured JSON when stdout is not a TTY (consistent with existing `--format json` behavior). `--args-json` does not imply non-interactive mode. TTY detection continues to govern interactive behavior as usual. A user invoking `--args-json` from an interactive terminal gets the same interactive experience (editor prompts, confirmation dialogs) as normal CLI usage. **Stdin interaction.** When `--args-json -` is used, the pre-parse logic consumes stdin to read the JSON payload. This means commands that also read stdin (e.g. `Query` piping input) will find it empty. JSON callers must embed all input in the JSON payload (e.g. the `query` field) rather than relying on stdin piping. This should be documented in the CLI help text for `--args-json`. ### Bridging Types An audit of all ~30 command structs identifies the following categories of field types and the work required for each. #### Types that already work Primitives (`bool`, `String`, `Option`, `u8`, `usize`, `Option`, `Option`), `Option`, and `schemars::Schema` already have or trivially gain `Serialize`, `Deserialize`, and `JsonSchema`. No changes needed beyond adding derives to the containing struct. #### Fields with custom `value_parser` functions Four fields use custom `value_parser` functions to translate CLI string input into typed values: `string_or_path` (→ `String`), `parse_schema` (→ `schemars::Schema`), `parse_fork_turns` (→ `Option`), and `parse_duration` (→ `DateTime`). These do not need bridging types. The target types already implement `Deserialize` and `JsonSchema`. Clap uses the `value_parser` to convert CLI strings; serde deserializes from JSON directly into the target type. The `value_parser` functions remain CLI-only and are invisible to serde. For example, `Query::schema` is `Option`. On the CLI path, clap applies `string_or_path.try_map(parse_schema)` to convert the DSL string. On the JSON path, serde deserializes a JSON Schema object natively. #### Simple enums `CliFormat`, `LogFormat`, and the two `Sort` enums (in `conversation/ls` and `conversation/grep`) are `clap::ValueEnum` types. They need `Serialize`, `Deserialize`, and `JsonSchema` added — one derive line each. `ReasoningConfig` already has `Serialize`/`Deserialize` via schematic's `Config` derive and needs only `JsonSchema`. #### FromStr types Seven types (`Editor`, `AttachmentUrlOrPath`, `WorkspaceIdOrPath`, `ConversationTarget`, `KvAssignment`, `ExpirationDuration`, `KeyValueOrPath`) already implement `FromStr`. They need `Deserialize` (delegating to `FromStr` for string inputs) and `JsonSchema` (emitting `{"type": "string"}` with appropriate descriptions or enum values). A shared helper trait or macro can reduce boilerplate across these. #### Manual clap types Three types have hand-written `clap::Args` / `FromArgMatches` impls: - **`ToolDirectives`**: An ordered list of tool enable/disable operations. Needs manual `Deserialize` and `JsonSchema` impls representing the same concept as a JSON array of directive objects. Because these types represent lists (not maps), they must not use `#[serde(flatten)]` — they are regular named fields in the serde representation. - **`FlagIds` and `PositionalIds`**: Lists of conversation targets. For JSON, both collapse to `Vec` — the positional-vs-flag distinction is a CLI-only concern. Need manual `Deserialize` and `JsonSchema` impls. Same caveat about not using `#[serde(flatten)]`. #### Triple-state fields (`Option>`) Nine fields across `Query`, `Edit`, `Fork`, and `Print` use clap's `num_args = 0..=1` with `default_missing_value` to represent three states: absent, flag-present-without-value, flag-present-with-value. For the JSON path, the three states map to: field absent or `undefined`, `null` or `true`, and a concrete value. These fields need custom serde handling to distinguish absent from null. The exact mechanism needs prototyping. Options include a `TriState` newtype that works as a clap field type, per-field `#[serde(default, deserialize_with)]` annotations, or a combination. Clap's derive macro has special handling for `Option>` tied to `num_args`, so a replacement type must either replicate that behavior or keep `Option>` for clap and use serde attributes for the JSON path. This will be resolved during Phase 1 implementation. #### `Commands::External` The `#[command(external_subcommand)]` variant currently holds `Vec`. Clap's `external_subcommand` requires a type implementing `FromIterator`. For JSON, the raw vector is not ergonomic — a structured object with `name` and `args` fields is better. The variant becomes `External(ExternalCommand)`, where `ExternalCommand` is a struct that implements `FromIterator` (for clap), `Serialize`, `Deserialize`, and `JsonSchema` (for the JSON path). For `--args-schema`, installed plugins that implement [RFD D19]'s `Help` protocol contribute their structured arg definitions to the schema as `oneOf` variants under the `External` discriminator. Plugins that don't implement `Help` get an opaque `{"name": "string", "args": ["string"]}` fallback. [RFD D19]'s `PluginArg` type needs a small extension for this: an optional `value_type` field (`"string" | "integer" | "number" | "boolean"`, defaulting to `"string"`) so the schema can emit proper JSON Schema types for plugin arguments. ### Schema Versioning The schema version (`$jp.schema_version`) tracks breaking changes to the JSON input contract — field renames, removed fields, structural changes to command nesting. It is independent of JP's release version. Adding new optional fields is not a breaking change and does not bump the schema version. Removing or renaming fields is breaking. Changing a field's type is breaking. The `schema_version` field in the input JSON is optional. When present, JP validates it matches the expected version and returns an error on mismatch. Validation happens in the envelope struct before the inner payload is deserialized into `Cli`. ## Drawbacks **Annotation overhead.** Every command struct gains three additional derives and some fields need `#[serde(...)]` / `#[schemars(...)]` attributes. This adds visual noise to the arg definitions. In practice, most structs need only the derive line — the handful of fields needing attributes are concentrated in `Query` and the conversation targeting types. **Dual maintenance of manual impls.** The three types with manual `clap::Args` impls (`ToolDirectives`, `FlagIds`, `PositionalIds`) now also need manual `Deserialize` and `JsonSchema` impls. These must be kept in sync with the clap behavior. Tests that round-trip CLI args through JSON can catch drift. **clap constraint gaps.** Clap's `conflicts_with` and `requires` constraints are not automatically enforced on the serde deserialization path. Invalid combinations (e.g. both `edit` and `no_edit` set to true) are caught by the execution pipeline rather than at parse time. This is acceptable for the initial implementation; a `validate()` pass can be added incrementally. **Binary size.** `schemars` adds code generation for schema derivation. The impact is expected to be modest since `schemars` is already a dependency (used by `Query::schema`), but should be measured. ## Alternatives ### Generate schema from clap introspection at runtime Walk `clap::Command` at runtime using its introspection API (`get_subcommands`, `get_arguments`, etc.) to produce a JSON Schema. Rejected because clap's introspection is lossy — value parsers are opaque, so the schema would emit `"string"` for nearly every argument. The core motivation for the schema is to give LLMs precise type information, which requires the richer type data that `schemars::JsonSchema` provides. ### Dedicated parallel schema structs Maintain a separate set of structs annotated only with `schemars` that mirror the CLI shape, with a conversion layer between them. Rejected because two sources of truth means every CLI change requires a corresponding schema struct update. This maintenance burden grows with every new command and flag. ### JSON → synthetic argv → clap parsing For `--args-json`, convert the JSON object into a `Vec` argv and feed it through clap's normal parsing path. This is simpler to implement but lossy in the other direction — some JSON structures (nested objects for `--cfg`, ordered arrays for `ToolDirectives`) have no natural argv representation. It also means every JSON input is subject to clap's string-based parsing quirks. Direct deserialization into the struct is more robust and preserves types end-to-end. ## Non-Goals - **Full clap constraint enforcement in JSON mode.** Constraints like `conflicts_with` and `requires` are not replicated in the `Deserialize` path. The execution pipeline catches real conflicts at runtime. - **Shell completions from the schema.** The JSON Schema could in theory generate shell completions, but that is out of scope. See [RFD 059]. - **Non-interactive mode.** `--args-json` does not imply non-interactive behavior. See [RFD 049] for the `--non-interactive` flag. - **`--cfg` JSON objects.** Accepting JSON objects as `--cfg` values is a useful extension to the config pipeline but orthogonal to the JSON CLI interface. It is tracked separately. - **Long-running JSON API server.** The schema defined here could serve as the request format for a persistent JSON-over-HTTP (or JSON-over-stdio) server that accepts JP commands without process-per-invocation overhead. That is a separate design concern, but this RFD's schema is intended to be reusable as the request contract for such a server. ## Risks and Open Questions 1. **schemars compatibility with schematic types.** Several CLI fields use types from `jp_config` that are derived via schematic's `Config` macro. These types have `Serialize`/`Deserialize` but may not have `JsonSchema`. Each needs to be checked — adding `JsonSchema` to schematic-derived types may require `#[schemars(with = "...")]` annotations or wrapper types. 2. **Schema stability.** Once external consumers depend on the schema, breaking changes become costly. The `schema_version` mechanism helps, but the initial schema should be designed carefully. Starting with `Query` as the first fully-schema'd command (the primary use case for LLM callers) allows iteration before committing to the full surface area. 3. **schemars version.** The project uses `schemars` 1.0.0-alpha.17. If the alpha API changes before 1.0 stable, derives and custom impls may need updating. 4. **Enum representation for `Commands`.** Serde's default externally-tagged enum produces `{"variant_name": {fields}}`. This is clean but the exact tag name and nesting for subcommands (e.g. `{"conversation": {"ls": {fields}}}`) needs to be validated against what schemars generates and whether LLMs handle nested discriminators well. 5. **Negative flags.** CLI has `--no-edit`, `--no-reasoning`, `--no-tool-use`, etc. In JSON, these are better expressed as the positive field set to `false` (e.g. `"edit": false` instead of `"no_edit": true`). The serde representation needs to decide whether to mirror the CLI naming or optimize for JSON ergonomics. Mirroring CLI naming is the default (since the structs are the source of truth), but `#[serde(rename)]` can alias where it matters. 6. **Triple-state clap interaction.** Clap's `Option>` has special derive macro support that a replacement type may not receive automatically. The exact bridging strategy (new type vs serde attributes vs hybrid) needs prototyping. ## Implementation Plan ### Phase 1: Bridging Types Create the types and annotations that command structs depend on: - Triple-state handling for `Option>` fields (prototyping needed to determine: `TriState` newtype, per-field `#[serde(deserialize_with)]`, or hybrid approach) - `ExternalCommand` struct with `FromIterator`, `Serialize`, `Deserialize`, `JsonSchema` - `Deserialize` + `JsonSchema` impls for `FromStr`-based types (`ConversationTarget`, `AttachmentUrlOrPath`, `WorkspaceIdOrPath`, `Editor`, `KvAssignment`, `ExpirationDuration`) - `Deserialize` + `JsonSchema` impls for manual-clap types (`ToolDirectives`, `FlagIds`, `PositionalIds`) Can be reviewed and merged independently. No user-visible behavior change. ### Phase 2: Derive Additions — `Query` and `Globals` Add `Serialize`, `Deserialize`, `JsonSchema` to `Cli`, `Globals`, `RootOpts`, `Commands`, and `Query`. This covers the primary use case (LLM callers invoking `jp query`). Validate that `schemars::schema_for!(Cli)` produces a usable schema. Write round-trip tests: construct a `Query` from JSON, verify it produces the same `PartialAppConfig` as the equivalent CLI args. ### Phase 3: `--args-schema` and `--args-json` Add the pre-parse detection for both flags. `--args-schema` generates and prints the schema with embedded `schema_version` and `jp_version` metadata, with optional command-path pruning of the schema AST. `--args-json` deserializes the input via the envelope struct, validates schema version if present, and dispatches into `run_inner`. These two flags are shipped together — the schema defines the contract, the input consumes it. ### Phase 4: Remaining Commands Add derives to the remaining command structs incrementally: `Conversation` (and all subcommands), `Config`, `Attachment`, `Plugin`, `Init`. Each subcommand tree can be a separate PR. ### Phase 5: External Plugin Schema (depends on [RFD D19]) Extend `PluginArg` with `value_type`. On `--args-schema`, query installed plugins via the `Help` protocol and inject their schemas into the `External` variant. ## References - [RFD 049: Non-Interactive Mode and Detached Prompt Policy][RFD 049] - [RFD 059: Shell Completions and Man Pages][RFD 059] - [RFD 072: Command Plugin System][RFD 072] - [RFD D19: Structured Plugin Help Protocol][RFD D19] [RFD 049]: ../049-non-interactive-mode-and-detached-prompt-policy.md [RFD 059]: ../059-shell-completions-and-man-pages.md [RFD 072]: ../072-command-plugin-system.md [RFD D19]: D19-structured-plugin-help-protocol.md