Jean-Pierre

Appearance

View Markdown
References039053064066078D05

RFD D33: Conversation Store and Bare Forking ​

Summary ​

This RFD adds conversation.store, an untyped IndexMap<String, JsonValue> on ConversationConfig, and jp conversation fork --no-turns, a fork variant that copies config without conversation events. Together these enable tools to persist arbitrary data that travels with a conversation and orchestrated workflows to chain conversations where each phase inherits accumulated state from its predecessor.

Both features build on the config mutation mechanism from RFD 078. The store is a config field — tools write to it via access.config grants and outcome.config. Turn-less forking copies the resolved config (including the store) into a fresh conversation.

Motivation ​

RFD 078 gives tools the ability to read and write config paths. But AppConfig has no general-purpose data field — every path is a typed config setting with specific semantics. Tools that need to persist arbitrary workflow data (a list of locked decisions, a research summary, a section tracker) have nowhere to put it without overloading an existing config field.

conversation.store fills this gap. It is an untyped map on ConversationConfig — a designated place for tool-produced data that has no meaning to JP's core systems but travels with the conversation through the full config lifecycle (persistence, rollback, forking, CLI seeding).

Turn-less forking addresses a related need: orchestrated workflows that span multiple conversations. RFD D05's RFD authoring pipeline runs explore, converge, and draft as separate conversations. Each phase needs the accumulated data from prior phases but a fresh message history. Regular forking copies both config and events; --no-turns copies only config.

Design ​

conversation.store ​

A new field on ConversationConfig:

rust
pub struct ConversationConfig {
    // ... existing fields ...
    pub store: IndexMap<String, serde_json::Value>,
}

The store is an untyped map. Values are opaque JSON. Tools use it to persist arbitrary data that travels with the conversation.

Because it lives on AppConfig, it gets the full config lifecycle:

  • File defaults: .jp/config.toml or persona files can pre-populate store keys.
  • CLI initialization: jp q --cfg 'conversation.store.rfd_id:="D32"' seeds the store for a run.
  • ConfigDelta persistence: tool writes via RFD 078's outcome.config are converted to ConfigDeltas. Rollback reverses them.
  • Forking: regular fork and bare fork both copy the store.

Tools access the store through access.config grants scoped to conversation.store.* or more specific paths like conversation.store.rfd.decisions.

Orthogonal Fork Opt-Outs ​

What this RFD originally bundled as a single --bare flag is decomposed into orthogonal opt-out flags, each suppressing one inherited aspect of a fork:

  • --no-turns: copy the source's full effective config but no conversation events — a fork with zero turns.
  • --no-parent: do not record parent-child lineage in the conversation tree.
  • --no-title: do not inherit the source's title.

The flags compose freely and apply to any fork: a regular fork can drop only parentage (--no-parent), and a turn-less fork keeps parentage unless --no-parent is also given. Bundling these under one flag conflated independent concerns — "no events", "no lineage", and "no title" are separate axes callers want to control separately.

--no-turns carries the event-collapse semantics originally proposed for --bare:

  • The source conversation's config layers (base config plus every ConfigDelta) are resolved into a single effective config.
  • That resolved config becomes the new conversation's base config.
  • No events are copied — the new conversation has zero turns.

The result is structurally identical to a conversation started fresh with the source's current effective config.

--no-parent defers to conversation-tree lineage (RFD 039); it is a no-op until parentage is actually tracked. --no-title defers to RFD 053, which owns the title lifecycle. A --bare convenience alias (implying --no-turns --no-parent) can be added later if the multi-phase workflows below want a single shorthand, but is not required.

This enables orchestrated workflows where each phase starts fresh but inherits accumulated data:

explore conversation (store accumulates research)
  → fork --no-turns --no-parent →
converge conversation (inherits research, accumulates decisions)
  → fork --no-turns --no-parent →
draft conversation (inherits research + decisions, produces RFD)

Example: RFD Authoring Pipeline ​

The rfd_decision tool is configured with store access:

toml
[conversation.tools.rfd_decision]
enable = true
run = "ask"
access.config.read = ["conversation.store.rfd.*"]
access.config.write = ["conversation.store.rfd.decisions"]

During the converge phase, the tool accumulates decisions:

json
{
  "content": "Decision #4 locked.",
  "config": {
    "conversation": {
      "store": {
        "rfd": {
          "decisions": [
            { "number": 1, "text": "Flat event structs.", "status": "locked" },
            { "number": 4, "text": "Chrome verbosity API.", "status": "locked" }
          ]
        }
      }
    }
  }
}

After the converge conversation ends, fork --no-turns --no-parent creates the draft conversation. The draft conversation's tools can read conversation.store.rfd.decisions to see the locked decisions from the prior phase.

Drawbacks ​

  • Config as data store feels unusual. AppConfig is traditionally "how the app is configured," not "data the app accumulates." Mitigated by namespacing under conversation.store.

  • ConfigDelta size. Tools that write large values to the store create large deltas. RFD 066 addresses content-addressable blob storage for large values.

  • Opting out of lineage hides the chain. --no-parent (used by the pipeline so each phase is independent) means the conversation tree cannot trace the relationship between the fork and its source, so jp conversation show won't display the chain. This is opt-in, not implied by --no-turns.

Alternatives ​

Dedicated store outside config ​

A separate persistence layer for tool data with custom events and rollback.

Rejected — duplicates existing config infrastructure. See RFD 078's Alternatives for the full argument.

Filesystem-based state directory ​

Use docs/rfd/.state/<NNN>/ (as originally proposed in RFD D05) for workflow data, with tools writing files directly.

Still viable as a complement for human-readable artifacts. But for machine-readable state that tools consume programmatically, the config-based store is more integrated (rollback, forking, inspectability via jp config show).

Fork with event filtering instead of --no-turns ​

Instead of --no-turns, add --from=start --until=0 or similar to copy config with an empty event range.

Rejected because the existing --from/--until flags operate on turn boundaries within the event stream. "Zero events" is a degenerate case that deserves its own flag with clear semantics rather than being expressed as an edge case of range filtering.

Non-Goals ​

  • Schema enforcement on store values. Values are opaque JSON. Typed schemas are future work.
  • Cross-conversation store access. Reading another conversation's store without forking is deferred. Bare forking is the only cross-conversation data path.
  • Store-aware compaction. RFD 064's compaction does not need special handling for store deltas — they are regular ConfigDeltas and compact using the same rules.

Risks and Open Questions ​

  • Store size limits. Should there be a cap on total store size per conversation? Tools that accumulate large artifacts could bloat the config. RFD 066 mitigates for individual large values but not for many small keys.

  • Bare fork discoverability. If a user bare-forks 10 conversations in a pipeline, there is no visible chain linking them. Metadata (e.g., a forked_from field in the store or conversation metadata) could help but is not proposed here.

Implementation Plan ​

Phase 1: conversation.store field ​

Add store: IndexMap<String, serde_json::Value> to ConversationConfig. Wire through config merge/delta/serialization. Verify CLI seeding, file defaults, rollback, and regular forking.

Depends on: RFD 078 Phase 1 (access.config grants, so tools can actually use the store).

Phase 2: fork --no-turns (implemented) ​

Add --no-turns to jp conversation fork: resolve the source's effective config (base plus every ConfigDelta) into the new conversation's base config and create an empty event stream. Shipped ahead of the rest of this RFD — the flag exists on jp conversation fork today.

--no-parent is deferred until conversation-tree lineage is tracked, and --no-title is deferred to RFD 053.

Can be merged independently of Phase 1.

References ​