Most code-intelligence tools benchmark against each other. A more useful question is what the tool adds over an agent that just greps and reads files. Modern coding agents are good with shell tools, so the honest baseline is a bare agent rather than a rival product.

We ran the same agent (Codex, gpt-5.6-sol) on 15 real code-navigation tasks across five pinned repositories, in paired runs: one arm with only grep / read_file / list_files, one arm with the published Miller v1.22.1 archive in its default configuration. Prompts, scoring, and the tool-call and token budgets (enforced by a recording proxy) were the same in both arms. Then we doubled both budgets to rule out starvation.

The numbers on this page come from the 2026-08-25 rerun on Miller v1.22.1: same 15 tasks, same five pinned commits, same model, same seed, same budgets as the first run on 2026-07-29. Three things about the method did change, and they are stated in the finding: the bare-agent adapter was rewritten from its recorded spec because the original source was lost, the harness gained a void-retry limit, and a defect in the recording proxy that could void a task pair was fixed before these runs. The bare arm is therefore a rebuilt baseline, not a byte-identical rerun of July's.

Five of the fifteen tasks need exact symbol identity: who references this specific function rather than everything sharing its name, the call path between two symbols, proof that a rename touches every site. Grep has no way to answer those, and the bare agent went 0 for 5 in every bare run of both measurements, at both budgets. On the ten tasks that don't need identity, Miller wins 7 to 5 at each budget.

The bare agent often finds the right evidence and then fumbles it. Counting evidence found regardless of answer quality, the arms are close: 0.72 for bare against 0.77 for Miller at the frozen budget, and 0.65 against 0.77 at the doubled budget. The rewritten baseline adapter finds more evidence than July's did (0.72 against July's 0.59 to 0.66) and still gets the same five tasks right. That strengthens the conclusion. The bare agent's problem is not finding evidence. It is exact identity, and acting correctly on what it found. Miller is also not the cheap option. It spends more calls and tokens per task than a baseline that fails fast, and that spend is what buys the correct answers.

The lexical core carries most of the result, and the semantic question got its own experiment. After a single run showed the semantic arm losing two tasks at the frozen budget, we reran that off-versus-on pair nine times overnight. The answer is noise: the lexical arm won 4 reps, the semantic arm won 4, one tied, and the mean difference was 0.2 tasks against a run-to-run scatter of 1.9. On this task set, at tight budgets, semantic retrieval neither helps nor hurts the score. That closes the do-no-harm question for the default; it does not measure semantic's value, because only 2 of the 15 tasks ask the concept-worded questions semantic retrieval was built for. MILLER_SEMANTIC=off remains a supported switch, and lexical output is byte-identical either way.