# Introducing PF Terminal
Published: 2026-07-28 · Post Fiat Research
PF Terminal is an open-source, multi-model coding terminal built on Codex. In seven matched benchmark comparisons, it used less model spend and less wall time than Hermes or Claude Code.
Canonical: https://postfiat.org/research/introducing-pf-terminal/
Markdown source of the page above. Other formats and the research blog index: https://postfiat.org/blog/
---
A coding harness determines how often a model runs, how much context it rereads, how many tool loops it completes, and how much you pay.
PF Terminal is designed to reduce that overhead while preserving access to strong models from multiple providers.
Across seven matched comparisons in our first-party release benchmark:
- PF Terminal's three-run coding-model spend was lower in **7/7 comparisons**. Savings ranged from **5.7% to 80.9%**, with a **39.8% median**.
- PF Terminal's three-run wall time was lower in **7/7 comparisons**. Its speed advantage ranged from **11.1% to 323.3%**, with the other harness taking a median **1.47× as long**.
- PF Terminal completed all three runs in **7/7 comparisons**; the matched baseline did so in **5/7**.
The PF Terminal team designed and ran the study. Each comparison used three PF Terminal runs and three baseline runs. No independent replication exists yet, so these results describe the frozen versions, tasks, and routes we tested rather than predicting every future run. Six comparisons use Hermes; one uses Claude Code.
## What is PF Terminal?
[PF Terminal](https://github.com/agtico/PfTerminal) is an open-source fork of Codex.
[Codex](https://github.com/openai/codex) is OpenAI's open-source coding agent. In a dated comparison of major terminal agents distributed through npm, Codex ranked first by package downloads. It provides an excellent terminal experience, but it natively supports OpenAI models. Its agent orchestration therefore stays within the OpenAI model family rather than routing work among providers.
[Hermes](https://openrouter.ai/apps/hermes-agent) is OpenRouter's most-used public application by attributed token volume and its top-ranked coding agent. It gives developers broad model choice, but our measurements showed a heavier execution loop on several matched tasks.
PF Terminal fills the gap between them. It keeps the Codex foundation and terminal workflow while adding multi-model routing and a leaner execution path.
| PF Terminal provides | Practical benefit |
| --- | --- |
| Multiple providers | Use configured Anthropic, OpenAI, OpenRouter, Vercel AI Gateway, Z.AI, and other routes from one terminal. |
| Explicit routing | Choose the model and provider for each run. |
| Model-aware orchestration | Assign planning, implementation, review, and visual work according to model capability and cost. |
| Local resumable sessions | Keep transcripts, workspaces, diffs, and session state on the machine where the code lives. |
| Live accounting | Track route, elapsed time, retries, token telemetry, and settled provider cost where available. |
| Open-source code | Inspect prompts, adapters, cache behavior, and orchestration logic. |
To install PF Terminal on Linux or from a macOS terminal, download the installer, inspect it, and then run it:
```sh
curl --fail --location --proto '=https' --tlsv1.2 \
--output pfterminal-install.sh \
https://github.com/agtico/PfTerminal/releases/latest/download/install.sh
less pfterminal-install.sh
sh pfterminal-install.sh
```
The installer downloads the platform-specific release archive and verifies its SHA-256 digest before installation. macOS users can instead download the DMG from the [GitHub releases page](https://github.com/agtico/PfTerminal/releases).
## Cost and speed: PF Terminal versus other harnesses
Every row below is one task and model route tested three times in each harness. The cost and time columns show the **average for one run**, calculated across those three attempts. Costs are the provider's settled bill for the coding model. Failed and timed-out attempts remain in the averages.
**Cost improvement** compares average cost per run. **Speed improvement** compares average elapsed time per run: other-harness time divided by PF Terminal time, minus one. Higher percentages favor PF Terminal. Percentages use unrounded run data, so they can differ slightly from the displayed dollar and minute averages.
### PF Terminal versus Hermes
6 runs per comparison3 PF Terminal + 3 Hermes. Cost and time are averages per run.
Task
Model / route
PF Terminal
Hermes
PF improvement
Cost / run
Time / run
Cost / run
Time / run
Cost saved
Speed gain
EventForgeImplementation
GLM 5.2OpenRouter
$0.05
6.6 min
$0.15
9.6 min
65.7%
46.6%
EventForgeImplementation
Kimi K3OpenRouter
$0.38
8.7 min
$0.54
10.0 min
29.0%
14.0%
QueueCraftRepository repair
GLM 5.2OpenRouter
$0.32
5.3 min
$0.34
5.9 min
5.7%
11.1%
QueueCraftRepository repair
GLM 5.2Vercel
$0.35
2.6 min
$0.58
4.4 min
39.8%
65.7%
QueueCraftRepository repair
Kimi K3OpenRouter
$0.31
2.4 min
$1.60
10.2 min
80.9%
323.3%
PF Terminal websiteGeneration
Kimi K3OpenRouter
$1.75
24.7 min
$2.52
31.2 min
30.7%
26.7%
### PF Terminal versus Claude Code
6 runs in this comparison3 PF Terminal + 3 Claude Code. Cost and time are averages per run.
Task
Model / route
PF Terminal
Claude Code
PF improvement
Cost / run
Time / run
Cost / run
Time / run
Cost saved
Speed gain
PF Terminal websiteGeneration
Opus 5Anthropic
$2.89
17.0 min
$6.70
29.7 min
56.9%
74.3%
The GLM 5.2 website attempt is omitted because two Hermes runs left the assigned image route, so it did not produce a valid harness-to-harness cost or speed comparison. The [complete report](/research/pfterminal-benchmark-0124/REPORT.md) retains those runs for route-adherence auditing.
### A separate Opus 5 coding suite
Before the release campaign, we ran Opus 5 through PF Terminal and Claude Code on three repository tasks: QueueCraft, TextWright, and QueryForge. Each task ran three times in each harness with fresh workspaces and a dedicated Anthropic API key per lane. Hidden checks verified the implementations and confirmed that the agents left the tests unchanged.
Both harnesses passed all nine runs. Anthropic Admin Usage attributed **$5.03** to PF Terminal and **$11.22** to Claude Code. PF Terminal therefore used **55.1% less model spend**, while Claude Code took **3.80× as long** on the overall median run.
6 runs per task3 PF Terminal + 3 Claude Code. The final row summarizes 9 runs per harness.
Task
PF Terminal
Claude Code
PF improvement
Cost / solve
Median time
Cost / solve
Median time
Cost saved
Speed gain
QueueCraftRepository repair
$0.53
73.0 sec
$1.15
214.6 sec
54.1%
194.0%
TextWrightRepository repair
$0.54
72.9 sec
$1.19
278.7 sec
55.0%
282.3%
QueryForgeRepository repair
$0.61
128.2 sec
$1.40
395.1 sec
56.1%
208.2%
All tasks9 runs per harness
$0.56
73.3 sec
$1.25
278.7 sec
55.1%
280.2%
The dollar figures in the task rows come from client telemetry. The lane totals and overall cost-per-solve figures come from Anthropic Admin Usage grouped by the two dedicated API-key IDs; client telemetry differed from the Admin total by $0.00022 across PF Terminal's nine runs and matched it exactly for Claude Code.
### PF Terminal versus Codex
PF Terminal adds multi-model support to Codex. That wider model choice should come without sacrificing Codex's performance on OpenAI models.
We compared PF Terminal with Codex on three coding tasks using the same `gpt-5.6-sol` model. Both passed every run. At the median, PF Terminal cost less on all three tasks and finished faster on two:
30 balanced runs5 runs per task in each harness; 15 PF Terminal + 15 Codex.
| Task | PF Terminal cost versus Codex | PF Terminal speed versus Codex |
| --- | ---: | ---: |
| QueueCraft | **8.5% lower** | **35.0% faster** |
| TextWright | **5.3% lower** | **5.4% faster** |
| QueryForge | **7.8% lower** | 13.0% slower |
The result is straightforward: PF Terminal's multi-model additions preserved Codex-level OpenAI performance in this test, with QueryForge latency still available to improve.
Codex develops quickly, so this comparison cannot be a one-time launch exercise. We will rerun the same parity suite as we adopt upstream Codex changes. Our standard is simple: PF Terminal users should gain access to more models without giving up the speed, cost, or reliability they would expect from Codex. The [full results and methodology](/research/pfterminal-benchmark-0124/codex-parity/REPORT.md) are public. Three additional PF Terminal QueueCraft runs, all successful, are reported separately as a cache-robustness check and do not affect the balanced comparison above.
## Why the same model can cost more in another harness
A coding harness repeatedly sends context to a model, receives output, executes tools, adds the results to the conversation, and decides whether another model call is needed.
Small differences in that loop accumulate quickly:
- **Extra model calls** add inference time and billing.
- **Larger repeated context** makes the model reread more tokens.
- **More generated output** raises cost, especially on models with expensive output tokens.
- **Retries and discarded artifacts** charge for work that never reaches the final result.
- **Serial tool loops** add latency one step at a time.
The Opus website traces illustrate the effect. Across three waves, PF Terminal used 125,898 output tokens; Claude Code used 315,093, or 2.50× as many. Claude Code also read 17,675,208 cached input tokens compared with 6,628,785 for PF Terminal. A cache-read token is previously processed context reused at a discount, but it still has a cost and takes part in each serial inference.
Claude Code reported 59, 71, and 76 model turns. Its third run reached the $8 budget ceiling before producing a successful agent exit. PF Terminal completed its three runs with individual agent costs between $2.82 and $3.00.
The Kimi QueueCraft traces show a similar pattern:
- PF Terminal made **32 model/API calls**; Hermes made **127**.
- PF Terminal exposed **811,008 cache-read tokens**; Hermes exposed **5,322,323**.
- PF Terminal generated **13,309 output tokens**; Hermes generated **29,024**.
- Hermes recorded **193 tool calls**.
Both harnesses solved all three workspaces. Hermes simply took a longer route to the same functional result.
These traces suggest why the measured differences occurred: fewer calls, less repeated context, less output, and fewer serial detours. They do not isolate the contribution of each factor. Controlled ablations of prompts, tool-result compaction, cache policy, retry policy, and turn limits remain future work.
Two relevant implementation choices are directly inspectable. PF Terminal's Anthropic request builder places stable cache markers on system, tool, and recent-user blocks while enforcing Anthropic's four-marker limit ([implementation and tests](https://github.com/agtico/PfTerminal/commit/b1f93596c4c97189988a0be347e9b94ba42f3951)). Its Kimi path evaluates continuation at the provider's actual completion boundary instead of treating partial streamed output as a finished turn ([implementation and regression tests](https://github.com/agtico/PfTerminal/commit/e2735093e)). These choices are consistent with the traces; this benchmark did not measure their individual causal effects.
## What did the websites look like?
Each visual agent received the same frozen product brief and starter repository. The task required an original responsive PF Terminal site, a working installation interaction, an interactive orchestration demo, accessible controls, and exactly three final GPT Image 2 assets.
These are benchmark outputs rather than the official PF Terminal website.
### Opus 5: PF Terminal versus Claude Code
Both wave-three sites used Opus 5 through direct Anthropic and passed the browser verifier. In balanced blind judging, the sites scored 97–97 in one ordering; PF Terminal scored 96 versus 93 after the A/B order was reversed. Because the two orders produced different winners, the wave was recorded as inconclusive.
Click either screenshot to open the full 1440 × 1000 capture.
PF Terminal + Opus 5Average per run: $2.89 · 17.0 minutesClaude Code + Opus 5Average per run: $6.70 · 29.7 minutes
Across all three Opus waves, the visual result was a draw: one PF Terminal win, one Claude Code win, and one order-sensitive result recorded as inconclusive. The clear difference was execution efficiency, rather than visual dominance.
Including confirmed image outputs, PF Terminal’s all-in cost was 52.4% lower in this comparison.
### Kimi K3: PF Terminal versus Hermes
These wave-one sites used Kimi K3 through OpenRouter. The PF Terminal site scored 98 versus 90 and 97 versus 90 after the order was reversed.
Click either screenshot to open the full 1440 × 1000 capture.
PF Terminal + Kimi K3Average per run: $1.75 · 24.7 minutesHermes + Kimi K3Average per run: $2.52 · 31.2 minutes
PF Terminal won all three Kimi visual waves under the balanced-order rule. It also cost 30.3% less after confirmed image outputs were included.
## Appendix: methodology, limits, and evidence
### Why Codex and Hermes are the baselines
We chose the comparison harnesses for demonstrated adoption.
OpenRouter's public [Hermes Agent profile](https://openrouter.ai/apps/hermes-agent) ranked Hermes first globally and first among coding agents when checked on July 28, 2026. OpenRouter also [describes Hermes as its most-used application by token volume](https://openrouter.ai/blog/tutorials/hermes-agent/). This measures attributed OpenRouter traffic, rather than all agent usage: the [ranking methodology](https://openrouter.ai/docs/agent-sdk/typescript/api-reference/datasets) excludes hidden and private apps, and token totals can span different provider tokenizers.
For terminal agents published on npm, we queried npm's download-count API for the fixed seven-day period from July 18 through July 24, 2026:
| Terminal agent package | npm downloads |
| --- | ---: |
| `@openai/codex` | [**15,226,164**](https://api.npmjs.org/downloads/point/2026-07-18:2026-07-24/%40openai%2Fcodex) |
| `@anthropic-ai/claude-code` | [10,760,218](https://api.npmjs.org/downloads/point/2026-07-18:2026-07-24/%40anthropic-ai%2Fclaude-code) |
| `@google/gemini-cli` | [567,566](https://api.npmjs.org/downloads/point/2026-07-18:2026-07-24/%40google%2Fgemini-cli) |
| `@qwen-code/qwen-code` | [97,147](https://api.npmjs.org/downloads/point/2026-07-18:2026-07-24/%40qwen-code%2Fqwen-code) |
Codex ranked first in this comparison at about 1.42 times Claude Code's package downloads. The package pages identify [Codex](https://www.npmjs.com/package/%40openai/codex), [Claude Code](https://www.npmjs.com/package/%40anthropic-ai/claude-code), [Gemini CLI](https://www.npmjs.com/package/%40google/gemini-cli), and [Qwen Code](https://www.npmjs.com/package/%40qwen-code/qwen-code) as the measured distributions. [npm downloads count package retrievals](https://blog.npmjs.org/post/92574016600/numeric-precision-matters-how-npm-download-counts-work.html), not unique developers, and the comparison excludes agents primarily distributed through other channels. It supports the narrower claim that Codex led this major npm-distributed terminal-agent set during the measured week.
### Study design
The campaign used:
- PF Terminal 0.1.24, product commit `81a6ff2f953ef5463e69e018e3c9515d0bd19ca3`;
- Claude Code 2.1.197;
- Hermes Agent 0.19.0, upstream commit `d83e858507a9bdb7f96c7a163d89c34c60909dcf`;
- Python 3.12.3 on the same Linux host.
Within each comparison, the harnesses used the same named model and provider route: direct Anthropic `claude-opus-5`, OpenRouter `moonshotai/kimi-k3`, OpenRouter `z-ai/glm-5.2`, or Vercel AI Gateway `zai/glm-5.2`.
Runs started in fresh workspaces with isolated agent homes. Lane order alternated by wave. Failed and timed-out runs remained in solve, spend, and elapsed-time denominators.
QueueCraft used visible tests, a hidden verifier, seven hidden bug probes, and checks that agents left the tests unchanged. EventForge used a held-out verifier after the agent exited. The website task used a Playwright verifier for desktop and mobile rendering, interactions, accessibility-related controls, asset rules, overflow, and browser errors. No generated site received manual repairs.
GPT-5.6-Sol judged each website pair twice with labels hidden and A/B order reversed. A wave received a winner only when both orders selected the same site; otherwise it was inconclusive.
Two Hermes GLM website waves were excluded from matched-route success comparisons. In those isolated terminals, Hermes removed the supplied OpenAI key and discovered other image routes: ChatGPT Codex OAuth in one wave and Vercel AI Gateway in another. Their time and spend remain in the denominators, but no comparable GLM all-in cost ratio is reported.
Provider billing supplied contestant costs. Total attributed campaign spend was $60.73 for agents, a confirmed $11.33 image-output lower bound, and $6.12 for neutral judging, before GPT Image 2 prompt-input charges.
Three waves per comparison establish replication within this campaign, rather than statistical significance. Results are a dated measurement of these frozen versions and routes.
The published 0.1.24 binary was rebuilt after the benchmark to add bounded retry to the macOS DMG packaging script. No Rust runtime, provider, caching, model, or orchestration code changed.
### Public benchmark package
The [sanitized benchmark package](/research/pfterminal-benchmark-0124/README.md) contains the full matrix, methodology, per-run data, judgments, and audits.
Primary scripts:
- [Visual-site runner](/research/pfterminal-benchmark-0124/scripts/run_visual.py)
- [QueueCraft runner](/research/pfterminal-benchmark-0124/scripts/run_queuecraft.py)
- [EventForge runner](/research/pfterminal-benchmark-0124/scripts/run_eventforge.py)
- [Balanced blind-vision judge](/research/pfterminal-benchmark-0124/scripts/judge_visual.py)
- [Anthropic cost snapshot](/research/pfterminal-benchmark-0124/scripts/snapshot_anthropic_costs.py)
- [Result summarizer](/research/pfterminal-benchmark-0124/scripts/summarize.py)
- [Secret scanner](/research/pfterminal-benchmark-0124/scripts/scan_secrets.py)
Key evidence:
- [Full runbook](/research/pfterminal-benchmark-0124/RUNBOOK.md)
- [Benchmark matrix](/research/pfterminal-benchmark-0124/MATRIX.json)
- [Human-readable report](/research/pfterminal-benchmark-0124/REPORT.md)
- [Machine-readable summary](/research/pfterminal-benchmark-0124/summary.json)
- [Per-run CSV](/research/pfterminal-benchmark-0124/summary.csv)
- [Blind judgments](/research/pfterminal-benchmark-0124/visual/blind_judgments.json)
- [Image-attempt audit](/research/pfterminal-benchmark-0124/visual/image_generation_audit.json)
- [Route-conformance audit](/research/pfterminal-benchmark-0124/visual/lane_conformance_audit.json)
- [Release provenance](/research/pfterminal-benchmark-0124/RELEASE_EVIDENCE.md)
- [Zero-hit secret scan](/research/pfterminal-benchmark-0124/secret_scan.json)
- [OpenAI/Codex parity backtest](/research/pfterminal-benchmark-0124/codex-parity/REPORT.md)
- [OpenAI/Codex parity machine summary](/research/pfterminal-benchmark-0124/codex-parity/summary.json)
The package also includes the [frozen visual prompt](/research/pfterminal-benchmark-0124/visual/prompt.md), QueueCraft [task](/research/pfterminal-benchmark-0124/tasks/queuecraft/bugged/BENCHMARK_TASK.md) and [verifier](/research/pfterminal-benchmark-0124/tasks/queuecraft/verifier/verify.py), and EventForge [task](/research/pfterminal-benchmark-0124/tasks/eventforge/BENCHMARK_TASK.md) and [hidden verifier](/research/pfterminal-benchmark-0124/tasks/eventforge/hidden_verifier.py).
