RFD 134: Pending Turn
DiscussionDesign2026-11-08Jean Mertz
Required byRFD 135
Summary
A pending turn is a marker at the end of a conversation stream that precedes entries recorded after the last finished turn. When the next turn starts, start_turn converts the marker into that turn's TurnStart in place, so between-turn entries belong to the turn they precede rather than the turn before them.
Motivation
A conversation stream is a sequence of turns, each opened by a TurnStart event. Because an entry recorded between two jp query invocations has no turn of its own, it is appended to the tail of the previous turn:
jp config set -i <id>appends a config delta after the last turn's final response (config/set.rs:95).jp query --cfg …records its config delta beforestart_turnappends the newTurnStart(query.rs:557-563), placing the new turn's config delta at the end of the preceding turn.
Config resolution is unaffected because config deltas apply in stream order and, as Global entries, survive turn pruning. However, workspace projection (assign_turn_indices) and any external tool that groups events.json by TurnStart assign those deltas to the previous turn, attributing a config change to a turn it never affected.
For provider-visible entries recorded between turns, attaching them to the previous turn breaks turn-based operations. Consider a stream A, X, B, where X is recorded after turn A as input for turn B:
--fork 1retains turnBand dropsXalong with turnA, leavingB's response referencing input missing from the fork.- Compacting turn
A(RFD 064) summarizesXintoAeven thoughBremains uncompacted. - Recording
Xafter a compaction that already covers the last turn placesXinside the compacted turn range, hiding it from the model.
System notifications queued between turns (RFD 011) have this exact lifecycle. Without a dedicated boundary for between-turn entries, features must either attach them to the previous turn or use a global scope that ignores turn position.
Design
The marker
[TS ReqA RespA] [P Δ X] between turns
[TS ReqA RespA] [TS Δ X ReqB RespB] after `jp query` starts turn BTS is a TurnStart, P is the pending marker, and Δ is a config delta.
The marker is EventKind::PendingTurn(PendingTurn), an empty struct serialized as {"type": "pending_turn", "timestamp": …}. Like TurnStart, it is turn-scoped and not provider-visible.
A stream contains at most one pending marker. Only entries recorded between turns follow it: Global entries (config deltas, compaction overlays, and patch overlays) and turn-scoped entries added as pending inputs for the next turn.
Writing
ConversationStream::open_pending() appends a PendingTurn marker unless the stream already ends with a pending section. Callers that record entries between turns call open_pending() before appending their entries.
ConversationStream cannot open the pending marker automatically inside add_config_delta: turns have no end marker, so the stream cannot distinguish an active turn from the gap between turns. A config delta written during a running turn (such as an inquiry writing to its stream copy at inquiry.rs:291, or tool config mutation in RFD 078) belongs to the running turn. add_config_delta therefore continues to append directly to the end of the stream, and between-turn callers explicitly call open_pending() first:
jp query, before recording its--cfgand editor config deltas.jp config set -i.- Plugin
querydelegation (dispatch.rs:1355).
Starting a turn
start_turn converts an existing PendingTurn marker into a TurnStart in place: the entry retains its event ID and stream position, updates its timestamp to the current time, and appends the new ChatRequest after the pending entries. When the stream has no PendingTurn marker, start_turn appends a new TurnStart at the end of the stream.
Reading
iter_turns,iter_events_by_turn, andturn_countend the last turn at thePendingTurnmarker and do not count the pending section as a turn.ConversationStream::pending()returns the slice of entries following the marker.- Workspace projection numbers pending entries one index past the last turn, keeping them outside every compaction range; turn summary generation walks
TurnStartentries and therefore skips the pending section. retain_last_turnscountsTurnStartentries and retains every entry after the cutoff, so--fork Npreserves both the lastNturns and any trailing pending entries.retain_turns, used byjp conversation fork --first Nand other turn-range selectors (fork.rs:184-185), associates the pending section with the last turn. If the range retains the last turn, the pending entries are kept with it. If the range drops the last turn, turn-scoped pending entries are removed, while thePendingTurnmarker is retained if any Global entries follow it so those config deltas remain attached to the fork's next turn. Without that rule, dropping the preceding turn would also drop thePendingTurnmarker and leave its config deltas attached to an earlier turn they never affected.sanitizeinserts aTurnStartat the start of a stream that has noChatRequest(stream.rs:1168-1191). RecognizingPendingTurnas a turn boundary preventssanitizefrom inserting an empty turn ahead of a stream that contains only pending entries.jp conversation printrenders the pending section after the last turn under a "pending" heading (instead of a turn number) whenever the turn selection includes the last turn, or when the conversation has no turns yet. Becauseprintrenders conversation events rather than config deltas, the pending section produces no output until turn-scoped pending events are added. Pluginread_eventsreturnsPendingTurnlike any other event.- A stream that holds only pending entries has
turn_count() == 0and is treated as a new conversation when deciding whether to generate a title on the first query. BecauseConversationStream::is_emptycounts thePendingTurnmarker as an event, checkingis_emptyafter recording a--cfgdelta (injp queryor pluginquerydelegation) would otherwise skip title generation.
Replay
trim_last_turn removes entries starting from the last turn's first ChatRequest to the end of the stream (rather than starting from its TurnStart), and converts that TurnStart back into a PendingTurn marker. Config deltas and patch overlays are retained, matching existing behavior. A compaction overlay is retained only if its range ends before the replayed turn; any overlay whose range reaches the replayed turn is dropped, matching ConversationStream::retain (RFD 097). Otherwise, a compaction summary covering the replayed turn would still cover the resulting pending section and replace the replayed request on the next turn (tracked for trim_last_turn in T-1hbzetp).
before: [TS ReqA RespA] [TS X1 ReqB RespB] [P X2]
replay: [TS ReqA RespA] [P X1]
then: [TS ReqA RespA] [TS X1 ReqB' RespB']X1 was recorded for turn B and remains pending for the replayed turn, whereas X2 was recorded after turn B and is discarded with B. Any stored summary on B's TurnStart is cleared when the entry reverts to a PendingTurn marker.
trim_trailing_empty_turn follows the same rule: a turn without a ChatRequest reverts to a PendingTurn marker, and a PendingTurn marker with no entries after it is removed.
Retrying an unanswered request
jp query --no-edit re-sends the last ChatRequest when it is the final event of its turn, and sends "continue" otherwise (query.rs:929-937). Because a failed query followed by jp config set -i leaves [TS ReqB] [P Δ] and PendingTurn is a turn-scoped entry, three rules change:
- Retry inspection examines the last started turn:
last_turn_eventstops at anyPendingTurnmarker and returns the last turn-scoped entry before it, soReqBis still selected and re-sent. - Deferred removal deletes the selected request by event ID instead of popping the last entry.
jp query --no-edit --cfg …records its config delta (and opens thePendingTurnmarker) after selectingReqBand before removing it; popping the tail entry would hit the marker, leaveReqBin place, and sendReqBa second time. - Removing the request also drops any compaction overlay whose range reaches that turn, matching replay.
Converting the now-requestless turn back into a PendingTurn marker and merging the trailing PendingTurn marker into it preserves the earlier marker position and keeps all pending entries in stream order:
before: [TS ReqA RespA] [TS ReqB] [P Δ]
remove: [TS ReqA RespA] [P Δ]
then: [TS ReqA RespA] [TS Δ ReqB' RespB']Repair
A stream containing two PendingTurn markers, or a TurnStart after a PendingTurn marker, is invalid. ConversationStream::from_parts repairs both shapes on load alongside event ID normalization (RFD 097) rather than failing:
- If a
TurnStartappears after aPendingTurnmarker,from_partsmoves theTurnStartto the marker's position and removes the marker, preserving the turn's event ID and summary while folding the pending entries into that turn. - Any second
PendingTurnmarker is removed.
The first case occurs when a pre-RFD 134 jp binary writes to a stream that has a PendingTurn marker: the older binary preserves pending_turn as an unknown entry (entry.rs:191-205) and appends its next TurnStart after it.
Drawbacks
- In-place entry kind mutation. Starting a turn mutates a
PendingTurnentry into aTurnStartin place. AlthoughConversationStreamalready mutates entries in place (set_turn_summaryedits aTurnStartand trim operations remove entries), no existing operation changes an entry's kind, so a caller holding the marker's event ID observes aTurnStartat that ID once the turn starts. - Manual marker opening. Callers that write between turns must call
open_pending()explicitly. A between-turn writer that omits the call appends its entries to the previous turn without an error. - Transient invalid state across binary versions. Between a write by a pre-RFD 134
jpbinary and the next load by an updated binary, the stream on disk holds aTurnStartafter aPendingTurnmarker until load-time repair normalizes it. - Shifted turn numbering for config deltas. Workspace projection and external tools that group
events.jsonbyTurnStartassign between-turn config deltas to the turn after them rather than the turn before them.
Alternatives
Wrap pending entries in a container entry. Rejected. Every consumer (print, jp conversation grep, plugin read_events, config resolution, and event ID bookkeeping) would have to descend into the container or silently miss pending entries, and starting a turn would have to unpack the entries while preserving their event IDs.
Represent a pending turn as an open TurnStart without a ChatRequest. Rejected. Every turn-level operation (compaction, summary generation, start_turn) would have to check whether the final TurnStart has a request yet, and --fork 1 immediately after a between-turn write would keep only the requestless turn. Using a distinct PendingTurn entry makes the unstarted state explicit in the entry type rather than inferred from missing events.
Give each between-turn entry its own TurnStart. Rejected. Requestless turns would inflate turn counts across the codebase, and --fork 1 would keep only the trailing between-turn entry without the turn that preceded it.
Use Global scope for between-turn inputs. Rejected. Global entries preserve stream position, but they survive every turn trim (--replay on turn B would retain inputs recorded after B) and are skipped by compaction ranges.
Leave between-turn entries in the previous turn. Rejected. This is the current behavior and causes the fork and compaction bugs described in Motivation.
Non-Goals
- New between-turn event types. This RFD introduces the
PendingTurnboundary; config deltas are the only existing entries that use it. - Mid-turn notification delivery. Between-turn notifications from RFD 011 can use the pending section once RFD 011 lands, but delivering notifications during a running turn requires its own mechanism.
- Storing unsent query drafts in the stream. Drafts such as
QUERY_MESSAGE.mdor replies queued during a running turn are user-local working state, whereas the conversation stream records committed events.
Risks and Open Questions
Preserving the marker's event ID on TurnStart. Retaining the marker's event ID when converting PendingTurn into TurnStart makes the marker's ID become the turn's ID. Allocating a new ID for the TurnStart would invalidate any reference a plugin held to the marker's ID, so this design preserves the existing ID.
Overwriting the marker's timestamp. Converting PendingTurn into TurnStart replaces the marker's opening timestamp with the turn's start timestamp. Each pending entry retains its own timestamp, so the time at which pending entries were recorded is still preserved.
Auditing between-turn writers. The three writers listed in Writing cover all current between-turn callers of add_config_delta in jp_cli, but any future command that writes to a stream between turns must also call open_pending().
Implementation Plan
- Marker and stream API. Add
EventKind::PendingTurn, itsTYPE_TAGSentry andencode_eventarm,open_pending,pending, thestart_turnin-place conversion, turn iteration and counting updates, projection numbering,retain_turns, and unit tests covering stream shapes. This phase is independently mergeable because nothing writes a marker yet. - Trim, retry, and repair. Update
trim_last_turnto cut from the firstChatRequest,trim_trailing_empty_turnto revert an empty turn toPendingTurn,last_turn_eventto stop at the marker, retry request removal to delete by event ID and merge markers, andfrom_partsto repair invalid marker ordering on load. Bothtrim_last_turnand retry request removal drop compaction overlays whose range reaches the removed turn. This phase depends on Phase 1 and on T-1hbzetp. - Writers and display. Call
open_pending()injp query,jp config set -i, and pluginquerydelegation, and updatejp conversation printto render the pending section when its turn selection includes the last turn. Treat a pending-only stream as empty when deciding whether to generate a conversation title injp queryand pluginquerydelegation. This phase depends on Phase 2. - Documentation. Add a glossary entry for Pending Turn (noting that it has no request and is not counted as a turn), and update the Turn glossary entry and
TurnStartdoc comment to note that a turn's leading entries can be recorded before itsChatRequest. This phase depends on Phase 3.