# RFD 062: CLI Usage Tracking

- **Status**: Discussion
- **Category**: Design
- **Authors**: Jean Mertz <git@jeanmertz.com>
- **Date**: 2026-03-21

## Summary

Introduce a `usage.json` file that records how the CLI is used — which commands
are invoked, which flags are passed, and how often. This data enables features
that adapt to a user's workflow. The file is stored per-workspace and updated
atomically on every command invocation.

This data is **NOT shared with any external service**, it is purely local-only,
used to enhance the local JP experience.

## Motivation

JP currently has no memory of how users interact with the CLI across sessions.
Every invocation starts from the same state: the same config, the same field
ordering, the same defaults. But users develop patterns — they frequently use
`--model=opus`, they always set `--reasoning=auto`, they reach for the same
`--cfg` fields repeatedly.

Without usage data, features that want to adapt to these patterns have no signal
to work with. Conversation state captures the stream of events during
interactions with the assistant, but not how the user invoked the CLI to get
there.

This RFD establishes the infrastructure for recording CLI usage. It defines the
storage format, access patterns, write semantics, and integration points. The
initial implementation records argument usage per command. Future RFDs can extend
the schema to capture additional signals (query durations, token counts, etc.)
without changing the infrastructure.

## Design

### What to track

The usage file records data that **cannot be derived from existing state**. The
guiding principle: if you can reconstruct it from conversation history, or
config files, don't duplicate it here.

Initial tracked data:

| Data                        | Why                                          |
|-----------------------------|----------------------------------------------|
| Command invocations         | Which commands are used, how often, when     |
| Argument usage per command  | Which arguments are passed, with what values |

Data explicitly **not** tracked (derivable from other sources):

| Data                       | Derivable from                     |
|----------------------------|------------------------------------|
| Conversation count/history | Workspace conversation storage     |
| Model usage over time      | Conversation event streams         |
| Config field values        | Config files + conversation deltas |

Future extensions (not implemented in this RFD, but the schema accommodates
them):

| Data               | Potential use                            |
|--------------------|------------------------------------------|
| Error frequency    | Surfacing reliability issues             |
| Interrupt patterns | UX decisions (streaming vs tool Ctrl+C)  |

Note that **query durations** and **token counts** are not tracked here. These
are metadata that belong on conversation events here the infrastructure already
exists. The usage file tracks CLI surface-level patterns, not conversation-level
telemetry.

### Storage location

Usage data is stored **per-workspace** because usage patterns vary across
workspaces. A user working on a project that requires Google models will have
different flag patterns than one using Anthropic. The workspace-local file
provides the most relevant signal.

- **With workspace**: `$XDG_DATA_HOME/jp/workspace/<id>/usage.json`
- **Without workspace** (fallback): `$XDG_DATA_HOME/jp/usage.json`

The fallback applies when no workspace is available (e.g., future commands that
operate outside a workspace context).

The workspace user storage path (`Workspace::user_storage_path()`) already
exists and is used for other per-user, per-workspace data (like the active
conversation ID). Usage tracking follows the same pattern.

### Schema

The file is JSON. The top-level structure mirrors the CLI's subcommand tree:

```json
{
  "cli": {
    "last_used": "2026-07-21T10:30:00Z",
    "commands": {
      "query": {
        "last_used": "2026-07-21T10:30:00Z",
        "args": {
          "model": {
            "last_used": "2026-07-21T10:30:00Z",
            "count": 42,
            "values": {
              "opus": {
                "count": 30,
                "last_used": "2026-07-21T10:30:00Z"
              },
              "haiku": {
                "count": 12,
                "last_used": "2026-07-15T08:00:00Z"
              }
            }
          },
          "config": {
            "last_used": "2026-07-20T14:15:00Z",
            "count": 14,
            "values": {
              "assistant.tool_choice=auto": {
                "count": 5,
                "last_used": "2026-07-20T14:15:00Z"
              },
              "assistant.tool_choice=required": {
                "count": 2,
                "last_used": "2026-07-19T09:00:00Z"
              },
              "style.reasoning.display=full": {
                "count": 7,
                "last_used": "2026-07-18T09:00:00Z"
              }
            }
          },
          "reasoning": {
            "last_used": "2026-07-18T09:00:00Z",
            "count": 15,
            "values": {
              "auto": {
                "count": 10,
                "last_used": "2026-07-18T09:00:00Z"
              },
              "off": {
                "count": 5,
                "last_used": "2026-07-16T12:00:00Z"
              }
            }
          }
        }
      },
      "config": {
        "last_used": "2026-07-19T11:00:00Z",
        "args": {},
        "commands": {
          "show": {
            "last_used": "2026-07-19T11:00:00Z",
            "args": {
              "explain": {
                "last_used": "2026-07-19T11:00:00Z",
                "count": 3,
                "values": {
                  "assistant.model.id": {
                    "count": 2,
                    "last_used": "2026-07-19T11:00:00Z"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
```

#### Uniform argument shape

Every argument entry has the same structure — no argument gets special
treatment:

```rust
struct ArgUsage {
    last_used: DateTime<Utc>,
    count: u64,
    values: HashMap<String, ValueUsage>,
}

struct ValueUsage {
    count: u64,
    last_used: DateTime<Utc>,
}
```

Arguments are keyed by their **clap argument ID** — the Rust field name in the
derive struct (e.g. `model`, `no_edit`, `reasoning`). This is the canonical
identifier that is stable regardless of whether the user typed `-m`, `--model`,
or `--model=opus`. Short flags, long flags, and positional arguments all share
the same ID space.

The `values` key is always the raw string the user typed as the argument's
value. For `model`, that's `opus`. For `reasoning`, that's `auto`. For `config`
(`--cfg`), that's `assistant.tool_choice=auto` (the full `KEY=VALUE` string).

This uniformity means consumers don't need to know which arguments are
"special." Any argument-specific interpretation (like splitting `--cfg` values
on `=` to group by field path) happens on the read side.

#### Recursive command structure

The `commands` object is recursive — subcommands nest naturally:

```rust
struct CommandUsage {
    last_used: DateTime<Utc>,
    args: HashMap<String, ArgUsage>,
    commands: HashMap<String, CommandUsage>,
}
```

`jp config show --explain` records under `cli.commands.config.commands.show`.
`jp query --model=opus` records under `cli.commands.query`.

### Integration: `CliUsage` type

A new `CliUsage` type in `jp_cli` manages loading, updating, and persisting
usage data. It is added to `Ctx` so any command handler can record usage:

```rust
pub(crate) struct Ctx {
    pub(crate) workspace: Workspace,
    // ... existing fields ...
    pub(crate) usage: CliUsage,
}
```

#### Loading

`CliUsage::load` reads the file from the workspace's user storage path (or the
global fallback). If the file doesn't exist or fails to parse, it returns an
empty `CliUsage` — corrupted or missing usage data is never an error. A warning
is logged if the file exists but can't be parsed.

Note: the `Cli::parse()` split into `Cli::command().get_matches()` +
`Cli::from_arg_matches()` described below is shared infrastructure that [RFD
060][] also builds on. RFD 060 uses the retained `ArgMatches` for config explain
provenance; this RFD uses them for usage recording. Both operate on the same
single parse result.

```rust
impl CliUsage {
    pub fn load(workspace: Option<&Workspace>) -> Self {
        let path = Self::storage_path(workspace);
        match path.and_then(|p| Self::read_from(&p)) {
            Some(usage) => usage,
            None => Self::default(),
        }
    }

    fn storage_path(workspace: Option<&Workspace>) -> Option<Utf8PathBuf> {
        workspace
            .and_then(Workspace::user_storage_path)
            .map(|p| p.join("usage.json"))
            .or_else(|| {
                user_data_dir().ok().map(|p| p.join("usage.json"))
            })
    }
}
```

#### Recording

Commands record their argument usage via `CliUsage::record`:

```rust
impl CliUsage {
    /// Record that an argument was used with the given value.
    pub fn record_arg(
        &mut self,
        command_path: &[&str],
        arg_id: &str,
        value: Option<&str>,
        now: DateTime<Utc>,
    ) {
        let cmd = self.get_or_create_command(command_path);
        cmd.last_used = now;

        let entry = cmd.args.entry(arg_id.to_owned()).or_default();
        entry.count += 1;
        entry.last_used = now;

        if let Some(val) = value {
            let val_entry = entry.values.entry(val.to_owned()).or_default();
            val_entry.count += 1;
            val_entry.last_used = now;
        }
    }
}
```

The `command_path` is a slice like `&["query"]` or `&["config", "show"]`,
derived from the command's position in the clap hierarchy.

#### Recording pattern: automatic via `ArgMatches`

Rather than manually adding `record_flag` calls in each command's `run` method,
flag recording is automated using clap's `ArgMatches` API.

The entry point in `run_inner()` splits `Cli::parse()` into two steps to retain
access to the raw matches:

```rust
// Before (current):
let cli = Cli::parse();

// After:
let matches = Cli::command().get_matches();
let cli = Cli::from_arg_matches(&matches).unwrap_or_else(|e| e.exit());
```

This is clap's canonical approach for mixing the derive and builder APIs. It
performs a single parse — no value parsers run twice, no file-reading side
effects from parsers like `string_or_path` are duplicated.

A generic `record_from_matches` function walks the `ArgMatches` tree and
records every argument whose `value_source()` is `ValueSource::CommandLine`:

```rust
fn record_from_matches(
    usage: &mut CliUsage,
    command_path: &[&str],
    matches: &ArgMatches,
    now: DateTime<Utc>,
) {
    // Record each explicitly-provided argument.
    for id in matches.ids() {
        let arg_id = id.as_str();
        if matches.value_source(arg_id) != Some(ValueSource::CommandLine) {
            continue;
        }

        // get_raw returns the pre-value-parser OsStr values.
        match matches.get_raw(arg_id) {
            Some(raw_values) => {
                for val in raw_values.filter_map(|v| v.to_str()) {
                    usage.record_arg(command_path, arg_id, Some(val), now);
                }
            }
            None => {
                // Boolean flag with no value (e.g. no_edit).
                usage.record_arg(command_path, arg_id, None, now);
            }
        }
    }

    // Recurse into subcommands.
    if let Some((name, sub_matches)) = matches.subcommand() {
        let mut path = command_path.to_vec();
        path.push(name);
        record_from_matches(usage, &path, sub_matches, now);
    }
}
```

This approach eliminates Phase 3 entirely — no per-command manual recording
needed. Every flag on every command is automatically captured. New flags added
to any command are tracked without any usage-tracking code changes.

Global arguments (like `--cfg`, `--verbose`, `--persist`) are handled correctly:
clap propagates them to subcommand matches, and `value_source()` accurately
reports `CommandLine` only when the user explicitly passed them.

#### Persisting

Usage is saved at the end of `run_inner()`, after the command completes but
before the process exits. This happens alongside the existing workspace
persistence:

```rust
// In run_inner(), after Commands::run() and task_handler.sync():
ctx.usage.save();
```

`save()` is fire-and-forget — it logs warnings on failure but never propagates
errors. Usage tracking must never cause a command to fail.

### Atomic writes

The current `write_json` helper in `jp_storage` writes directly to the target
path, which can corrupt the file if the process is interrupted mid-write. Usage
tracking requires atomic writes because every command invocation writes to the
same file.

`CliUsage::save` writes atomically using temp-file-then-rename:

```rust
impl CliUsage {
    pub fn save(&self) {
        let Some(path) = Self::storage_path(self.workspace.as_ref()) else {
            return;
        };

        if let Err(error) = self.write_atomic(&path) {
            tracing::warn!(%error, path = %path, "Failed to save usage data.");
        }
    }

    fn write_atomic(&self, path: &Utf8Path) -> io::Result<()> {
        if let Some(parent) = path.parent() {
            fs::create_dir_all(parent)?;
        }

        // Write to a temp file in the same directory (same filesystem
        // guarantees atomic rename on POSIX).
        let dir = path.parent().unwrap_or(path);
        let mut tmp = tempfile::NamedTempFile::new_in(dir)?;
        serde_json::to_writer_pretty(&mut tmp, &self.data)?;
        tmp.as_file_mut().flush()?;
        tmp.persist(path)?;

        Ok(())
    }
}
```

`tempfile::NamedTempFile::persist` uses `rename(2)` on POSIX, which is atomic
within a filesystem. On Windows it uses `MoveFileEx` with
`MOVEFILE_REPLACE_EXISTING`.

### Concurrency

Multiple JP processes can run concurrently (e.g., detached queries, parallel
terminal sessions). Without locking, concurrent writes can race: both processes
read the file, both update their in-memory copy, and the last writer wins —
the first writer's updates are lost.

For the initial implementation, this is acceptable. The data is heuristic — a
lost increment of a flag counter doesn't affect correctness. The atomic write
ensures the file is never corrupted (partially written), only potentially
stale.

If this becomes a problem (e.g., detached queries become common and usage data
diverges noticeably), a future improvement could use advisory file locking
(`flock` on POSIX) with a read-modify-write cycle. This RFD does not implement
locking.

### Privacy

The usage file is **local-only**. It is never transmitted to any external
service. It exists solely to improve the local JP experience. This must be
documented clearly in user-facing docs and the `jp init` output.

The file contains no conversation content, prompts, or LLM responses — only
structural metadata about CLI invocations (command names, argument IDs,
argument values like model names or config field paths).

## Drawbacks

- **Write on every invocation**: Every command run writes to `usage.json`. This
  is a small I/O cost (~1-2ms for a JSON write + fsync + rename) but it's
  unavoidable overhead even when no feature consumes the data yet.

- **Stale data under concurrency**: Without locking, concurrent writes cause
  last-writer-wins races. Some usage increments will be silently lost. This is
  acceptable for heuristic data but means the counts are approximate, not
  exact.

- **Schema evolution**: The JSON schema will grow as more data is tracked. Old
  `usage.json` files need to be forward-compatible (ignore unknown fields) and
  the code must handle missing fields gracefully (default to zero/empty).

## Alternatives

### SQLite instead of JSON

SQLite provides atomic writes, concurrent access, and efficient queries out of
the box. However, it adds a non-trivial dependency, is harder to inspect
manually, and is overkill for what is essentially a small counters file. JSON
is human-readable, debuggable, and sufficient for the expected data size.

### No persistent tracking — session-only

Track usage only within a single session (in-memory). This avoids all file I/O
concerns but provides no cross-session signal. Features like wizard field
ordering need historical data to be useful.

### Append-only log instead of mutable JSON

Write one line per invocation to a log file, aggregate on read. This avoids
concurrent-write races (appends are atomic on POSIX for small writes) but
makes reads expensive (must scan the full log) and requires periodic
compaction. Not worth the complexity for counter data.

## Non-Goals

- **Analytics or telemetry**: This is not a telemetry system. No data leaves
  the user's machine. There is no aggregation server, no opt-in/opt-out
  toggle, no anonymization pipeline.

- **Query performance tracking**: Tracking query durations, token counts, or
  error rates is a natural extension of this infrastructure but is out of
  scope for this RFD. The schema is designed to accommodate it (additional
  top-level keys alongside `cli`), but the recording logic is not
  implemented here.

- **Cross-workspace aggregation**: Each workspace has its own `usage.json`.
  There is no mechanism to merge or query across workspaces. If a user wants
  global stats, that's a future feature.

## Risks and Open Questions

- **Schema versioning**: The `usage.json` file has no version field in this
  initial design. If the schema changes in a breaking way, we'd need to
  detect the old format and migrate or discard. Adding a `"version": 1` field
  now is cheap insurance — worth considering before this RFD is accepted.

- **Large value cardinality**: Flags like `--cfg` can have unbounded value
  diversity (every unique `KEY=VALUE` string becomes an entry). Over months
  of use, the `values` map could grow large. A future RFD could add eviction
  (e.g., drop entries older than 90 days or with count < 3), but this isn't
  needed initially.

- **`--no-persist` interaction**: When the user passes `--no-persist` (or
  `-!`), workspace persistence is disabled. Should this also suppress usage
  tracking? The argument for: consistency with the "leave no trace" intent.
  The argument against: usage is user-local metadata, not workspace state.
  Recommend: suppress usage writes when `--no-persist` is active, for
  consistency.

- **Field renames orphan usage data**: Clap's derive API assigns argument IDs
  from the Rust field name, not from `long` or `short`. Renaming a field
  (e.g. `model` → `model_id`) changes the clap ID from `model` to `model-id`
  even if the CLI flags (`-m`, `--model`) are unchanged. This orphans the old
  ID's counters and starts fresh under the new ID. This is more likely than a
  CLI-breaking rename since the field name is an internal detail. Mitigations:
  orphaned entries age out via eviction (described above), and the data is
  heuristic — a reset counter is a temporary accuracy loss, not a failure. The
  alternative — manual `record_arg` calls with hardcoded strings — would
  survive a rename only if someone remembers to update the string, and silently
  records under the wrong name if they forget. If stability becomes important,
  explicit `#[arg(id = "model")]` attributes could decouple the ID from the
  field name, but that adds boilerplate to every arg definition and isn't worth
  it initially.

## Implementation Plan

### Phase 1: Core types and storage

1. Define `CliUsage`, `CommandUsage`, `ArgUsage`, and `ValueUsage` types with
   serde derives.
2. Implement `CliUsage::load()` with graceful fallback on missing/corrupt
   files.
3. Implement `CliUsage::save()` with atomic temp-file-then-rename writes.
4. Implement `CliUsage::storage_path()` with workspace-local and global
   fallback logic.
5. Unit tests for load/save roundtrip, corrupt file handling, missing file
   handling.

### Phase 2: Integration into Ctx and run_inner

1. Add `usage: CliUsage` field to `Ctx`.
2. Load usage in `run_inner()` after workspace initialization.
3. Save usage at the end of `run_inner()`, gated on `persist` flag.
4. Implement `CliUsage::record_arg()` and
   `CliUsage::record_command_invocation()`.

### Phase 3: Wire up automatic recording in `run_inner`

1. Split `Cli::parse()` into `Cli::command().get_matches()` +
   `Cli::from_arg_matches()` in `run_inner()`.
2. Call `record_from_matches` with the root `ArgMatches` after workspace
   initialization.
3. All commands and flags are tracked automatically — no per-command changes.

Phase 1 can be merged independently. Phase 2 depends on Phase 1. Phase 3
depends on Phase 2 and is a single PR touching only `run_inner()`.

## References

- [RFD 061]: Interactive config (first consumer of usage data)
- `Ctx` in `jp_cli/src/ctx.rs` — the CLI context struct
- `Workspace::user_storage_path()` — per-user, per-workspace storage
- `user_data_dir()` in `jp_workspace` — global user data directory
- `write_json` in `jp_storage/src/value.rs` — current (non-atomic) JSON writer
- `tempfile::NamedTempFile::persist` — atomic rename on POSIX/Windows

[RFD 060]: 060-config-explain.md
[RFD 061]: 061-interactive-config.md