RFD 137: Stable Storage for Link Files
DiscussionDesign2026-07-09Jean Mertz
Summary
A link file is a disposable plain-text copy of conversation content that JP writes to disk so a printed path or file:// hyperlink has a target to open: the full tool result behind results_file_link, and the summary reported by jp conversation compact. This RFD moves link files from the OS temp directory to a per-conversation directory inside the User-Workspace Directory, names them by content hash, and prunes them by age. File contents and printed links are unchanged.
Motivation
Two code paths write link files into env::temp_dir():
ToolRenderer::render_resultwrites every rendered tool result totool_call_<millis>.<ext>, andresults_file_linkcontrols whether JP prints a filesystem path or an OSC 8 hyperlink to that file.jp conversation compactwrites each generated summary to<conversation_id>-summary-<from>-<to>.mdand prints the path in its timeline.
The tool result path has four defects:
- The OS controls file lifetime. The OS clears its temp directory on its own schedule, breaking links in terminal scrollback outside JP's control.
- Filenames collide.
<millis>is the sub-second component of the system clock (0..=999), repeating every second. A later tool result that lands on the same millisecond and extension overwrites the file an earlier link points to. - Failed writes still print a link.
render_resultignores the write result, so a printed link can point to a missing or stale file. - Unreferenced files are written. Even with
results_file_link = "off", rendering an inline tool result writes a temp file. Becausejp conversation printuses the same renderer, replaying a conversation rewrites every tool result file.
The summary path has a separate collision bug: its filename depends only on the conversation ID and turn range. Compacting a range, removing that compaction with jp conversation compact --reset, and compacting the same range again overwrites the file, so the earlier printed path opens the newer summary.
Design
User-Facing Behavior
Terminal output format is unchanged; only the underlying file path moves:
see: /home/user/.local/share/jp/workspace/jp-a1b2c/tmp/conversations/jp-c17457886043/fs_read_file-9f2c4e7a1b3d5f60.rsEach link remains valid for the retention window (30 days by default), measured from the last time JP printed a link to that file. jp conversation print reuses link files written during live runs and writes a file only when none exists for that content.
Storage Location
<user-workspace directory>/tmp/conversations/<conversation_id>/<label>-<hash>.<ext>The User-Workspace Directory (W-00dxybd#user-workspace-directory) is the workspace's per-user data directory (~/.local/share/jp/workspace/<slug>-<id>/ on Linux). JP resolves this directory at startup (Ctx::user_storage_path), and link file resolution simply appends a relative path to it. No methods are added to the storage backend traits (RFD 073) because link files are rendering artifacts rather than workspace data.
Storing link files in the User-Workspace Directory provides three properties:
- Outside the project tree. Link files stay out of version control and are invisible to tools that scan project files.
- Scoped to the workspace. The directory already holds the user-local conversations from which link files are derived (RFD 031). Command plugins that receive the user-workspace path can access the directory, though link files are not a plugin interface (see Relationship to
events.jsonand the Blob Store). - Managed by JP. JP owns the directory lifecycle and can prune expired files on its own schedule.
The tmp/ subdirectory holds expendable data: deleting it removes only link files. Storage operations do not preserve tmp/; when merging two user-workspace directories for the same workspace, JP keeps one tmp/ directory and may discard the other.
Nesting under tmp/conversations/<conversation_id>/ reserves tmp/ for other expendable files and groups link files by conversation. Because a conversation ID is immutable, link file paths survive title changes and archiving. Link files are not stored inside the conversation's own storage directory (see Alternatives).
File Naming
| Component | Value |
|---|---|
<label> | Sanitized tool name for a tool result, or summary for a compaction summary |
<hash> | First 16 lowercase hex characters of the content's SHA-256 digest |
<ext> | File extension detected by the writer, or omitted if none |
Because tool names come from model output, <label> restricts the tool name to ASCII alphanumeric characters, _, and -. If the tool name is unknown, <label> defaults to tool.
Content-addressed filenames have three consequences:
- Identical content shares one file. Replaying a conversation with
jp conversation printresolves to the file written during the live run. - Distinct content never collides. Distinct tool results get distinct filenames, as do distinct summaries of the same turn range.
- The renderer needs only the conversation ID. It does not need an event ID or operation index. A fanned-out tool call writes one file per operation during a live run and one file for the merged response on replay without index bookkeeping.
Writing Link Files
- JP writes a link file only when printing a path or hyperlink to it. Rendering a tool result with
results_file_link = "off"writes no file. Because the content remains in the event stream, a subsequent render that enables links (such asjp conversation print --style full) writes the file at that time. - If the target file already exists, JP updates its modification time (
mtime) without rewriting the content. Otherwise, JP writes to a temporary sibling file and atomically renames it into place so readers never observe a partial write. - If writing fails, JP logs a warning, omits the link, and still renders any inline result.
Link URLs (file:// and copy://) percent-encode path segments. Formatting a raw filesystem path into a URL breaks when the path contains spaces or non-ASCII characters, which occurs on macOS where the default user data directory is ~/Library/Application Support/jp and <slug> comes from the checkout directory name.
Plain paths printed by results_file_link = "full" are not escaped. Backslash-escaping spaces would help shell pasting but breaks terminal path auto-detection at the first backslash, whereas file:// URLs remain clickable as a single span.
OS Temp Directory Fallback
JP falls back to writing <conversation_id>-<label>-<hash>.<ext> in env::temp_dir() when:
- the command runs with
--no-persist, which must not write to JP's storage; or - the workspace has no User-Workspace Directory.
JP never prunes files in the OS temp directory.
Pruning
| Contract | Value |
|---|---|
| Config key | conversation.link_files.retention_days |
| Default | 30 |
| Never prune | "unlimited" |
| Rejected | 0 (fails validation with an error suggesting "unlimited") |
| Run point | End of every workspace-loading invocation, except --no-persist runs |
| Scope | Expired files under tmp/conversations/, then any directories left empty |
| On failure | Logs a trace warning without failing the command |
The pruning sweep deletes any link file whose mtime is older than the retention window. Because printing a link refreshes mtime, the retention window is measured from the most recent print rather than file creation.
The sweep also removes the link-file directory of any conversation that no longer exists, regardless of file age. To avoid deleting link files for a conversation created by a concurrent jp process after the current run loaded its index, the sweep re-scans stored conversation IDs (both live and archived) on disk, matching the session-mapping cleanup pass. Because jp conversation rm and temporary-conversation expiration run before the end-of-run cleanup, a deleted conversation's link files are removed in the same invocation. Any subdirectory under tmp/conversations/ whose name does not parse as a conversation ID is pruned by age only.
The sweep runs inside the existing end-of-run cleanup in jp_cli::run alongside orphaned lock-file and stale session-mapping removal, covering every command that writes link files.
0 is rejected because a zero-day window would delete newly written link files at the end of the run that created them; "unlimited" disables pruning.
Unlike RFD 130, which fixes session-record retention at 30 days, link-file retention is configurable because link files are plain-text copies of tool output that can grow large, and users differ on how they trade disk usage against scrollback link longevity.
Relationship to events.json and the Blob Store
Tool results can exist in three storage locations, each serving a distinct purpose:
| Location | Content | Purpose |
|---|---|---|
events.json | Persisted response, base64-encoded | Canonical conversation stream |
| Blob store (RFD 066) | Persisted payload, gzip-compressed | Offloads large payloads from events.json |
| Link file | Rendered output as plain text | Directly openable target for terminal links |
The blob store does not replace link files. Blobs are content-addressed and gzip-compressed so workspace text searches skip them, which prevents editors from opening them directly. Terminal links require an uncompressed file with an appropriate extension, whether the underlying payload lives inline in events.json or in the blob store.
Link files are disposable rendering artifacts, and the event stream remains the source of truth:
- A rendered result may have no link file. JP writes a file only when printing a link, so no file exists when links are disabled, the tool is hidden, or the write fails.
- Link files can be deleted at any time. JP prunes expired files automatically, and removing
tmp/manually is always safe. - Filenames and paths are not a programmatic interface. Both can change without notice, and nothing in JP reads link files back.
Features that consume tool output must read the event stream; features that require output omitted from the event stream must provide their own storage.
Code Organization
A LinkFiles type in jp_cli encapsulates the link-file directory (or OS temp fallback) for a single conversation and exposes one write method that returns Option<PathBuf>. Both ToolRenderer::render_result and jp conversation compact write through LinkFiles, and jp_cli::run invokes the sibling pruning function at the end of each run.
Drawbacks
- Plain-text copies persist longer.
events.jsonbase64-encodes tool content, whereas link files store plain text. Moving link files out of the OS temp directory keeps plain-text copies on disk for the full retention window (or until the conversation is removed). - Disk usage moves to the user data directory. Storage under the User-Workspace Directory grows with the volume of distinct tool results and summaries rendered within the retention window.
- Printed paths are longer and may contain spaces. On macOS, the default user data path contains a space (
Application Support). Underresults_file_link = "full", terminals that stop path auto-detection at whitespace only link the prefix before the space, and pasting the path into a shell requires quoting. Theosc8link style is unaffected. - End-of-run sweep overhead. Every persistent workspace invocation walks
tmp/conversations/and scans stored conversation IDs at exit.
Alternatives
- Keep using the OS temp directory with content-addressed filenames. This eliminates filename collisions, but the OS still deletes link files on its own schedule.
- Use the platform cache directory. System cache cleaners can purge it at any time, it is not scoped per workspace, and it introduces another JP-managed root directory to document.
- Store link files inside the conversation's storage directory. Conversation directory names include the title slug and change whenever the title is edited or auto-generated at the end of the first query, and archiving moves the directory under
.archive/. Either move breaks every previously printed link. In addition, storage synchronization and directory merges manage files inside conversation directories explicitly. (Version control is not an issue because link files would live only in the user-local conversation directory, while workspace projection copies only the three managed JSON files.) If user-local conversation directories later adopt a stable path that survives renames and archiving (T-1n0d3w0), link files can move there. - Place the retention setting under a top-level
[storage]section. A top-level section would contain only this single key;conversation.link_filesgroups it with the rest of the conversation settings. - Store link files under
.jp/in the project checkout. Placing link files inside the checkout exposes them to version control and project-scanning tools. - Create one directory per invocation. Because a link file depends only on its content, per-run directories would duplicate files across every
jp conversation printreplay. If JP later records per-run diagnostic manifests, those manifests can reference the content-addressed link files written during the run. - Name link files by the response's
EventId. Event-ID naming requires threading theEventIdinto the renderer and adding an operation index for fanned-out tool calls. It also gives each file a name that looks like a stable stream-entry address, inviting other features to depend on it. Content hashing avoids both costs and produces filenames with no meaning outside the link. - Emit custom-scheme links that resolve stream entries directly. A custom URI scheme handled by a user script requires no files on disk, but does nothing until the user configures a scheme handler.
file://links work out of the box in any terminal supporting OSC 8. - Link directly to the blob store. Rejected for the reasons in Relationship to
events.jsonand the Blob Store.
Non-Goals
- Defining a machine-readable schema for link files. Link files hold the human-readable text rendered by the writer;
events.jsonremains the machine-readable source of truth. - Durable payload storage. Durable storage belongs to the content-addressable blob store (RFD 066).
- Exposing link files to the assistant or other features. Link files are not a programmatic interface (see Relationship to
events.jsonand the Blob Store). - Preserving tool output omitted from the event stream. RFD 115 describes the renderer's temp file as retaining untruncated tool output beyond its persistence cap. This RFD makes no such guarantee (when
results_file_link = "off", JP writes no link file), leaving untruncated output storage to RFD 115. - Changing link rendering options. When links are printed, their display text, and
results_file_link(including its error overlay) remain unchanged. - Relocating diagnostic temp files. Trace logs and dumped request or config payloads in the OS temp directory have separate lifecycle requirements.
- Size-based cache eviction. Pruning is strictly age- and conversation-based.
Risks and Open Questions
- Workspace-wide pruning uses the active run's retention window. The end-of-run sweep prunes
tmp/conversations/across the entire workspace using the current invocation's resolvedretention_days. Running a conversation configured with a shorter retention window prunes other conversations' link files to that shorter window. - Conversations present only in another checkout's projection. If a conversation exists exclusively in checkout A's
.jp/conversations/and has not been imported into user-local storage, ajpinvocation in checkout B does not see that conversation ID during its sweep and deletes its link files. Existing links break early, though reprinting the conversation in checkout A recreates the files. - Directory walk cost at exit. The sweep stats every link file in the workspace on every run. If profiling shows noticeable latency, a timestamp marker file can throttle the age-based sweep to once per day.
- Filesystems without reliable
mtime. On filesystems that do not track modification times, link files may be pruned prematurely or never expire. - Sweep race with concurrent rendering. One process's sweep can read an expired
mtimejust before a concurrent process refreshes it and prints a link, unlinking the file right after the link is printed. The next render recreates the file. - Hash truncation collisions. Truncating SHA-256 to 16 hex characters yields a 64-bit space scoped to a single conversation directory, making collisions negligible at the number of tool results in a conversation.
Implementation Plan
- Add
LinkFiles. ImplementLinkFilesinjp_cliwith directory resolution, OS temp directory fallback, content-addressed naming, atomic write-if-absent via rename, andmtimerefresh. Unit-test against a temporary root with deterministic content and filenames. This phase can merge independently before any callers are wired up. - Migrate link-file writers. Route
ToolRenderer::render_resultandjp conversation compactthroughLinkFiles. Updaterender_resultto pass the tool name as the label, write a file only when emitting a path or link, suppress the link on write failure, and percent-encode link URLs. RetainLinkFilesonTurnRendereracross per-turnToolRendererrebuilds. Depends on Phase 1. - Add configuration and end-of-run pruning. Add
conversation.link_files.retention_days(with validation rejecting0) and wire the pruning sweep (including deleted-conversation cleanup) into the end-of-run cleanup injp_cli::run. Depends on Phase 1. - Update documentation. Add a "Link File" entry to the Ubiquitous Language glossary, document
tmp/under the User-Workspace Directory entry (W-00dxybd), and documentconversation.link_files.retention_days.
References
- RFD 031: Durable conversation storage and the User-Workspace Directory.
- RFD 066: Content-addressable blob store for durable payload storage.
- RFD 073: Layered storage backend traits (unchanged by this RFD).
- RFD 115: Bounded tool output, which describes the renderer's temp file as retaining output beyond its cap; this RFD does not guarantee that (see Non-Goals).
- RFD 130: Session record retention, which establishes the 30-day retention precedent.
crates/jp_cli/src/render/tool.rs:ToolRenderer::render_result.crates/jp_cli/src/cmd/conversation/compact.rs:write_summary_file.crates/jp_cli/src/lib.rs: End-of-run cleanup injp_cli::run.crates/jp_cli/src/ctx.rs:Ctx::user_storage_path.crates/jp_config/src/fs.rs: User data directory resolution.