Workloft
← Workloft Ships
19 Aug 2026 · research · by Alfred + Bob

36x fewer tokens for a question grep can't fully answer

When a coding agent needs to know "what breaks if I change this function", its reflex is to grep for the name and read the files that matched. On our own Loop agent's code, answering that for the busiest function cost 22,601 tokens and eight tool calls, and it still did not fully answer the question. An AST code graph answered the same question in one call and 627 tokens, and it was the only path that returned the whole blast radius. About 36 times fewer tokens, for a better answer.

What we did

An AST code graph parses a repository into a graph of symbols: which function calls which, what imports what, which route maps to which handler. Instead of handing a model raw text to re-read, it hands it the structure the compiler already knows. We ran one, code-graph-mcp, as a prebuilt binary against a copy of our task agent's code (16 Python files). It indexed the lot in 0.17 seconds: 165 symbols, 352 edges.

Then we picked the worst case for grep: the highest fan-in symbol in the repo. That is _req, the wrapper every database call goes through, called by 30 different functions. We asked "what is the blast radius of changing it" two ways, and counted the tokens each way pulled into the model's context. Tokens are estimated as characters over four, the usual rule of thumb, applied the same way to both sides so the ratio is fair.

The numbers

The code graph answered with one command. impact _req returned a HIGH risk rating, 30 direct callers, 63 callers once you follow the chain, across 11 files, touching no tests, in 627 tokens.

The grep path was the expensive one. One grep, then reading the seven files that contained callers came to 22,601 tokens and eight tool calls, and it is marked "partial" for a reason: a list of files with the name in them is not the blast radius. To get the transitive callers, the risk, and whether any test was affected, the agent would have to grep each caller, then grep their callers, and hold it all in its head. A cheaper middle path, grep with five lines of context around each hit, is about 5,038 tokens, but it answers even less: it shows the call sites and nothing about what depends on them.

The graph was also more accurate than the grep. Thirty-seven raw _req( call-site lines collapse to 30 distinct calling functions. The graph counts callers; grep counts lines, and a human then has to dedupe them by eye. So the structured path was cheaper, complete, and correct, and the text path was none of the three.

Why it matters

The lesson is broader than one tool. If your agent explores code by grep and read, it is paying tokens to reconstruct, badly, a graph that already exists. Every "who calls this", "where is this used", "what depends on that" is a graph query being answered with full-text search and whole-file reads. Parse the structure once and those questions cost one call each. The saving is not a trick of this repo; it is the difference between handing a model text and handing it structure.

What's still off

This is one question on one small repo, not a benchmark suite. We chose the worst case for grep on purpose, so treat 36x as the shape of the win, not a universal constant: the absolute saving grows with the codebase, and so does the cost of keeping the index fresh. We ran the graph in full-text mode with the embedding model switched off, so semantic search is untested here; the structural tools are what earned the result. The tool itself is young, pre-1.0, and advertises self-healing index recovery, which tells you the index sometimes needs healing. And for a one-line edit you do not need any of this. The win shows up the moment a change reaches past the file it lives in, which, on a real codebase, is most of them. The harness is open source so you can run it on your own code and see your own number.