Introducing PF Terminal

A coding harness determines how often a model runs, how much context it rereads, how many tool loops it completes, and how much you pay.

PF Terminal is designed to reduce that overhead while preserving access to strong models from multiple providers.

Across seven matched comparisons in our first-party release benchmark:

• PF Terminal’s three-run coding-model spend was lower in 7/7 comparisons. Savings ranged from 5.7% to 80.9%, with a 39.8% median.
• PF Terminal’s three-run wall time was lower in 7/7 comparisons. Its speed advantage ranged from 11.1% to 323.3%, with the other harness taking a median 1.47× as long.
• PF Terminal completed all three runs in 7/7 comparisons; the matched baseline did so in 5/7.

The PF Terminal team designed and ran the study. Each comparison used three PF Terminal runs and three baseline runs. No independent replication exists yet, so these results describe the frozen versions, tasks, and routes we tested rather than predicting every future run. Six comparisons use Hermes; one uses Claude Code.

What is PF Terminal?

PF Terminal is an open-source fork of Codex.

Codex is OpenAI’s open-source coding agent. In a dated comparison of major terminal agents distributed through npm, Codex ranked first by package downloads. It provides an excellent terminal experience, but it natively supports OpenAI models. Its agent orchestration therefore stays within the OpenAI model family rather than routing work among providers.

Hermes is OpenRouter’s most-used public application by attributed token volume and its top-ranked coding agent. It gives developers broad model choice, but our measurements showed a heavier execution loop on several matched tasks.

PF Terminal fills the gap between them. It keeps the Codex foundation and terminal workflow while adding multi-model routing and a leaner execution path.

To install PF Terminal on Linux or from a macOS terminal, download the installer, inspect it, and then run it:

curl --fail --location --proto '=https' --tlsv1.2 \
  --output pfterminal-install.sh \
  https://github.com/agtico/PfTerminal/releases/latest/download/install.sh
less pfterminal-install.sh
sh pfterminal-install.sh

The installer downloads the platform-specific release archive and verifies its SHA-256 digest before installation. macOS users can instead download the DMG from the GitHub releases page.

Cost and speed: PF Terminal versus other harnesses

Every row below is one task and model route tested three times in each harness. The cost and time columns show the average for one run, calculated across those three attempts. Costs are the provider’s settled bill for the coding model. Failed and timed-out attempts remain in the averages.

Cost improvement compares average cost per run. Speed improvement compares average elapsed time per run: other-harness time divided by PF Terminal time, minus one. Higher percentages favor PF Terminal. Percentages use unrounded run data, so they can differ slightly from the displayed dollar and minute averages.

PF Terminal versus Hermes

6 runs per comparison3 PF Terminal + 3 Hermes. Cost and time are averages per run.

PF Terminal versus Claude Code

6 runs in this comparison3 PF Terminal + 3 Claude Code. Cost and time are averages per run.

The GLM 5.2 website attempt is omitted because two Hermes runs left the assigned image route, so it did not produce a valid harness-to-harness cost or speed comparison. The complete report retains those runs for route-adherence auditing.

A separate Opus 5 coding suite

Before the release campaign, we ran Opus 5 through PF Terminal and Claude Code on three repository tasks: QueueCraft, TextWright, and QueryForge. Each task ran three times in each harness with fresh workspaces and a dedicated Anthropic API key per lane. Hidden checks verified the implementations and confirmed that the agents left the tests unchanged.

Both harnesses passed all nine runs. Anthropic Admin Usage attributed $5.03 to PF Terminal and $11.22 to Claude Code. PF Terminal therefore used 55.1% less model spend, while Claude Code took 3.80× as long on the overall median run.

6 runs per task3 PF Terminal + 3 Claude Code. The final row summarizes 9 runs per harness.

The dollar figures in the task rows come from client telemetry. The lane totals and overall cost-per-solve figures come from Anthropic Admin Usage grouped by the two dedicated API-key IDs; client telemetry differed from the Admin total by $0.00022 across PF Terminal’s nine runs and matched it exactly for Claude Code.

PF Terminal versus Codex

PF Terminal adds multi-model support to Codex. That wider model choice should come without sacrificing Codex’s performance on OpenAI models.

We compared PF Terminal with Codex on three coding tasks using the same gpt-5.6-sol model. Both passed every run. At the median, PF Terminal cost less on all three tasks and finished faster on two:

30 balanced runs5 runs per task in each harness; 15 PF Terminal + 15 Codex.

The result is straightforward: PF Terminal’s multi-model additions preserved Codex-level OpenAI performance in this test, with QueryForge latency still available to improve.

Codex develops quickly, so this comparison cannot be a one-time launch exercise. We will rerun the same parity suite as we adopt upstream Codex changes. Our standard is simple: PF Terminal users should gain access to more models without giving up the speed, cost, or reliability they would expect from Codex. The full results and methodology are public. Three additional PF Terminal QueueCraft runs, all successful, are reported separately as a cache-robustness check and do not affect the balanced comparison above.

Why the same model can cost more in another harness

A coding harness repeatedly sends context to a model, receives output, executes tools, adds the results to the conversation, and decides whether another model call is needed.

Small differences in that loop accumulate quickly:

• Extra model calls add inference time and billing.
• Larger repeated context makes the model reread more tokens.
• More generated output raises cost, especially on models with expensive output tokens.
• Retries and discarded artifacts charge for work that never reaches the final result.
• Serial tool loops add latency one step at a time.

The Opus website traces illustrate the effect. Across three waves, PF Terminal used 125,898 output tokens; Claude Code used 315,093, or 2.50× as many. Claude Code also read 17,675,208 cached input tokens compared with 6,628,785 for PF Terminal. A cache-read token is previously processed context reused at a discount, but it still has a cost and takes part in each serial inference.

Claude Code reported 59, 71, and 76 model turns. Its third run reached the $8 budget ceiling before producing a successful agent exit. PF Terminal completed its three runs with individual agent costs between $2.82 and $3.00.

The Kimi QueueCraft traces show a similar pattern:

• PF Terminal made 32 model/API calls; Hermes made 127.
• PF Terminal exposed 811,008 cache-read tokens; Hermes exposed 5,322,323.
• PF Terminal generated 13,309 output tokens; Hermes generated 29,024.
• Hermes recorded 193 tool calls.

Both harnesses solved all three workspaces. Hermes simply took a longer route to the same functional result.

These traces suggest why the measured differences occurred: fewer calls, less repeated context, less output, and fewer serial detours. They do not isolate the contribution of each factor. Controlled ablations of prompts, tool-result compaction, cache policy, retry policy, and turn limits remain future work.

Two relevant implementation choices are directly inspectable. PF Terminal’s Anthropic request builder places stable cache markers on system, tool, and recent-user blocks while enforcing Anthropic’s four-marker limit (implementation and tests). Its Kimi path evaluates continuation at the provider’s actual completion boundary instead of treating partial streamed output as a finished turn (implementation and regression tests). These choices are consistent with the traces; this benchmark did not measure their individual causal effects.

What did the websites look like?

Each visual agent received the same frozen product brief and starter repository. The task required an original responsive PF Terminal site, a working installation interaction, an interactive orchestration demo, accessible controls, and exactly three final GPT Image 2 assets.

These are benchmark outputs rather than the official PF Terminal website.

Opus 5: PF Terminal versus Claude Code

Both wave-three sites used Opus 5 through direct Anthropic and passed the browser verifier. In balanced blind judging, the sites scored 97–97 in one ordering; PF Terminal scored 96 versus 93 after the A/B order was reversed. Because the two orders produced different winners, the wave was recorded as inconclusive.

Click either screenshot to open the full 1440 × 1000 capture.

Across all three Opus waves, the visual result was a draw: one PF Terminal win, one Claude Code win, and one order-sensitive result recorded as inconclusive. The clear difference was execution efficiency, rather than visual dominance.

Including confirmed image outputs, PF Terminal’s all-in cost was 52.4% lower in this comparison.

Kimi K3: PF Terminal versus Hermes

These wave-one sites used Kimi K3 through OpenRouter. The PF Terminal site scored 98 versus 90 and 97 versus 90 after the order was reversed.

Click either screenshot to open the full 1440 × 1000 capture.

PF Terminal won all three Kimi visual waves under the balanced-order rule. It also cost 30.3% less after confirmed image outputs were included.

Appendix: methodology, limits, and evidence

Why Codex and Hermes are the baselines

We chose the comparison harnesses for demonstrated adoption.

OpenRouter’s public Hermes Agent profile ranked Hermes first globally and first among coding agents when checked on July 28, 2026. OpenRouter also describes Hermes as its most-used application by token volume. This measures attributed OpenRouter traffic, rather than all agent usage: the ranking methodology excludes hidden and private apps, and token totals can span different provider tokenizers.

For terminal agents published on npm, we queried npm’s download-count API for the fixed seven-day period from July 18 through July 24, 2026:

Codex ranked first in this comparison at about 1.42 times Claude Code’s package downloads. The package pages identify Codex, Claude Code, Gemini CLI, and Qwen Code as the measured distributions. npm downloads count package retrievals, not unique developers, and the comparison excludes agents primarily distributed through other channels. It supports the narrower claim that Codex led this major npm-distributed terminal-agent set during the measured week.

Study design

The campaign used:

• PF Terminal 0.1.24, product commit 81a6ff2f953ef5463e69e018e3c9515d0bd19ca3;
• Claude Code 2.1.197;
• Hermes Agent 0.19.0, upstream commit d83e858507a9bdb7f96c7a163d89c34c60909dcf;
• Python 3.12.3 on the same Linux host.

Within each comparison, the harnesses used the same named model and provider route: direct Anthropic claude-opus-5, OpenRouter moonshotai/kimi-k3, OpenRouter z-ai/glm-5.2, or Vercel AI Gateway zai/glm-5.2.

Runs started in fresh workspaces with isolated agent homes. Lane order alternated by wave. Failed and timed-out runs remained in solve, spend, and elapsed-time denominators.

QueueCraft used visible tests, a hidden verifier, seven hidden bug probes, and checks that agents left the tests unchanged. EventForge used a held-out verifier after the agent exited. The website task used a Playwright verifier for desktop and mobile rendering, interactions, accessibility-related controls, asset rules, overflow, and browser errors. No generated site received manual repairs.

GPT-5.6-Sol judged each website pair twice with labels hidden and A/B order reversed. A wave received a winner only when both orders selected the same site; otherwise it was inconclusive.

Two Hermes GLM website waves were excluded from matched-route success comparisons. In those isolated terminals, Hermes removed the supplied OpenAI key and discovered other image routes: ChatGPT Codex OAuth in one wave and Vercel AI Gateway in another. Their time and spend remain in the denominators, but no comparable GLM all-in cost ratio is reported.

Provider billing supplied contestant costs. Total attributed campaign spend was $60.73 for agents, a confirmed $11.33 image-output lower bound, and $6.12 for neutral judging, before GPT Image 2 prompt-input charges.

Three waves per comparison establish replication within this campaign, rather than statistical significance. Results are a dated measurement of these frozen versions and routes.

The published 0.1.24 binary was rebuilt after the benchmark to add bounded retry to the macOS DMG packaging script. No Rust runtime, provider, caching, model, or orchestration code changed.

Public benchmark package

The sanitized benchmark package contains the full matrix, methodology, per-run data, judgments, and audits.

Primary scripts:

• Visual-site runner
• QueueCraft runner
• EventForge runner
• Balanced blind-vision judge
• Anthropic cost snapshot
• Result summarizer
• Secret scanner

Key evidence:

• Full runbook
• Benchmark matrix
• Human-readable report
• Machine-readable summary
• Per-run CSV
• Blind judgments
• Image-attempt audit
• Route-conformance audit
• Release provenance
• Zero-hit secret scan
• OpenAI/Codex parity backtest
• OpenAI/Codex parity machine summary

The package also includes the frozen visual prompt, QueueCraft task and verifier, and EventForge task and hidden verifier.