A large benchmark of the RTK terminal-output compression tool found that reducing the text shown to coding agents did not consistently reduce the cost of completing software tasks. The results challenge claims that dramatic cuts in displayed output translate directly into equally dramatic savings on model tokens or bills.
Quesma tested RTK with Claude Code running Fable 5.0 and OpenCode running DeepSeek V4 Pro 0813 on Terminal-Bench 2.1. Each task was attempted five times with RTK and five times without it under the same routes and time limits. After removing four tasks that triggered refusals, the comparison covered 85 Fable tasks and 89 DeepSeek tasks, for 1,740 attempts.
On total cost, Fable attempts were 5% cheaper with RTK, while DeepSeek attempts cost 5% more. Pass rates also slipped slightly: one percentage point for Fable and two points for DeepSeek. When Quesma divided all spending by successful completions, Fable was 3% cheaper and DeepSeek 7% more expensive with the tool enabled.
A task-weighted analysis produced a less favourable picture. Fable was about 1% more expensive, with no clear difference from zero, while DeepSeek's average task cost increased 17%. Most of Fable's apparent savings came from one task that finished in roughly half as many turns with RTK. Removing that task reduced savings across the remaining work to less than 1%.
The gap between RTK's internal counter and billed usage was especially striking. Across 445 DeepSeek attempts, RTK reported 349.2 million tokens saved, an 89% reduction. But the counter estimates removed command output by comparing raw and filtered bytes; it does not measure billed tokens or account for changes in later agent turns. In one example, limited file-reading commands were compared with an entire file, producing very large estimated savings even though the original commands would never have returned the full content.
Compression can also change an agent's behaviour. One DeepSeek run entered a long error loop after RTK rewrote a command using an unsupported option. The issue was fixed in a later RTK release, but the attempt cost roughly nine times its matching baseline. Quesma said the broader trend remained even after excluding that outlier.
Caching and tool design further weaken the assumption that shorter shell output must mean a lower bill. Much terminal context is reread at discounted cache rates, and coding platforms often expose file-reading and search tools that bypass shell-output rewriting. Agents also commonly limit output themselves with commands such as head, tail or wc.
Quesma concluded that RTK may help particular workloads but should not be treated as a general cost-saving layer. The benchmark suggests teams should evaluate end-to-end task cost, completion quality and number of turns rather than relying on a compression counter alone.
