research-bot
Deep research on cheap tokens — worker models do the reading, the expensive model only ever sees the receipts.
The most expensive thing an AI agent can do is read. Point a premium model at a research question and it will happily burn its context — and your bill — on raw web pages, re-reading every fetched page on every subsequent turn. research-bot is a Python CLI built around a simple fix: let cheap worker models do all the reading, and let the expensive orchestrator only ever see condensed receipts.
How it works
The shape is three tiers. An architect (you, or a top-tier model) writes the intent
brief. Orchestrators break it into missions and dispatch workers. Workers are
research CLI processes — one per page — that search, fetch (free fetch first,
then paid Kagi Extract, then the Wayback Machine), and summarize to disk. The one
non-negotiable, straight from the README: receipts, not notes. Workers write full
distillates to files and return only a path plus a short RELEVANT LEADS block on stdout,
which is what keeps the orchestrator's context — and bill — small. Every LLM role
(summarizer, verifier, merger, planner, judge) is an AgentSpec in a registry,
swappable by env var or config file without touching code, and the filesystem is the
database: cached pages, an append-only usage ledger, grep as the query engine.
The 9.5× afternoon
The README's proven-shape number comes from one real afternoon: 4 missions, market research across 11 US states, run as 1 architect → 4 orchestrators → 262 workers. Total cost at published list rates: $7.88. The same token flow priced on premium APIs (orchestrator at Opus rates, workers at Sonnet rates) comes to $74.50 — 9.5× more for the same work. The repo is upfront that this is a cost comparison, not a quality claim: both sides are priced at list because, as the design notes put it, subscriptions don't run businesses.
The verification gate
Cheap summaries you can't trust are just cheaper noise, so the pipeline has a gate:
research verify notes.md re-reads each cited source from the page cache and
has a verifier agent adversarially audit every claim — SUPPORTED, DISTORTED, or
UNSUPPORTED, with quotes — then exits nonzero on FAIL so orchestrators can block on it.
Is that LLMs grading LLMs? The verify bench suite has hard ground truth: hand-planted
distortions frozen from real missions, scored on whether the verifier catches them.
And the gate earns its keep: on its first real outing it caught a misattributed quote
in production notes, which is why the playbook now requires a verify pass before any
load-bearing fact enters a deliverable.
Run it yourself
Python, MIT license, installs with uv. Only a DeepSeek API key is required for the
worker path; a Kagi key upgrades search and unlocks the Extract fetch tier. Cache
invalidation is "never," on purpose — any URL is paid for at most once, ever, so the
accounting stays exact and replayable. The full three-tier pattern is in
docs/playbook.md, and dated bench results across worker models are in
BENCHMARKS.md.
git clone https://github.com/eanderson4/research-bot cd research-bot uv tool install -e . # puts `research` on PATH research check # which keys are configured research summarize URL -o notes/ && research verify notes/<file>.md
The full README, verbatim
research-bot
Cheap-token deep research. A worker CLI (research) that does all the
expensive reading — web search, page fetch with bot-wall escalation, and
worker-model summarization — so the expensive model orchestrating the work
only ever reads condensed receipts. Plus a cross-tier token report that
prices the whole run against premium-API list rates.
Proven shape (one afternoon, 4 missions, 11 US states of market research):
1 architect → 4 orchestrators → 262 workers, $7.88 at list vs $74.50
for the same token flow on premium APIs — 9.5× cheaper. The playbook that
produced that is in docs/playbook.md, and the
methodology behind the number is in Design decisions
below.
Install
git clone https://github.com/eanderson4/research-bot
cd research-bot
uv tool install -e . # puts `research` on PATH
# or run in place:
uv run research check
Keys
First file found wins, environment variables override:
$RESEARCH_KEYS_FILE~/.config/research-bot/keys.json
{ "deepseek_api_key": "...", "zai_api_key": "...", "kagi_api_key": "...", "openrouter_api_key": "..." }
Only deepseek_api_key is required for the worker path. kagi_api_key
upgrades search (Kagi Search API, ~$25/1k) and unlocks the Kagi Extract
fetch tier ($4/1k pages); without it, search falls back to DuckDuckGo.
Commands
research check # which keys are configured
research search "QUERY" -n 8 # cached; repeat queries replay free
research fetch URL # free fetch -> Kagi Extract -> Wayback escalation
research summarize URL... -o DIR # summarizer agent writes notes to disk, stdout = receipt only
research verify NOTES.md # adversarial fact-check of notes vs their cited sources
research ask -p "PROMPT" # generic worker completion
research agents # the agent registry: roles, models, active overrides
research bench run --suite verify --model flash,pro # benchmark a role across models
research bench report # suite x model pivot of all bench results
research report --anthropic # cross-tier token report + list-price comparison
research serve # read-only localhost dashboard for live missions
Model aliases: flash (deepseek-v4-flash, default), pro (deepseek-v4-pro),
glm (glm-5.2 via z.ai or OpenRouter).
Agents
Every LLM role is an AgentSpec (system prompt + model + params + output
contract) in a registry (research_bot/agents.py); call sites never hardcode
prompts. Roles: summarizer, verifier, merger, planner, judge.
Swap any of them without code changes:
RESEARCH_AGENT_<ROLE>=pro— model override via env<store>/agents/<role>.md— replaces the system prompt<store>/agents/<role>.json— overrides any field (model, max_tokens, ...)
research agents shows what's resolved. Benchmarks live in
bench/ with cases frozen from real missions — run the
bench before promoting a model or prompt change to a role's default.
The bench measures task quality per role. Provider mechanics — TTFT,
sustained tok/s, cost per probe — are the companion repo
llm-meter; the two share result-field
conventions, and CI cross-checks that list rates and the schema haven't
drifted (tests/test_meter_compat.py).
A dated snapshot of bench results across worker models lives in
BENCHMARKS.md; raw per-run data is in
bench/results.jsonl.
Verification gate
research verify notes.md re-reads each cited source from the page cache
and has the verifier agent adversarially audit every claim
(SUPPORTED / DISTORTED / UNSUPPORTED, with quotes). Multi-source notes files
are split per ### <url> section so each claim is audited against its own
source. It writes notes.verdict.md and exits nonzero on FAIL (1 = claims
failed, 2 = a source couldn't be fetched), so orchestrators can gate:
research summarize $URL -o notes/ && research verify notes/<file>.md
On its first real outing the verifier caught a misattributed quote in production notes, so the playbook now requires this pass for load-bearing facts before they enter a deliverable.
Where things land
Root = $RESEARCH_ROOT, else the nearest enclosing .git directory, else cwd.
- searches/pages cache:
<root>/knowledge-base/research/{searches,pages}if that directory exists, else<root>/.research/{searches,pages}— file pairs (<slug>-<hash>.json+.md) plus anINDEX.md. The filesystem is the database, grep is the query engine. Paid calls are never replayed. - usage ledger:
<root>/.token-usage.jsonl(append-only; every search, extract page, and LLM call). SetRESEARCH_RUN=<mission>to tag records for per-run attribution, thenresearch report --run <mission>.
The one non-negotiable
Receipts, not notes. Orchestrating agents always call summarize -o so
full distillates go to disk and stdout carries only the path + RELEVANT
LEADS block. This is what keeps the orchestrator's context (and bill) small
— skipping -o silently leaks every page summary into the agent loop, where
it gets re-billed on every subsequent turn. See the playbook for the full
three-tier pattern.
Design decisions
Why compare at API list price? Because subscriptions don't run
businesses. A flat-rate coding plan is a promotional artifact, not a price
you can scale a production workload on, so the only stable basis for
comparing stacks is published per-token list rates on both sides. That's
what research report --anthropic does: the same token flow, priced at
list on both sides (orchestrator↔Opus, workers↔Sonnet), with the mapping and
rates in one place (research_bot/pricing.py). It is a cost comparison, not
a quality claim — quality is what the bench suites and the verification gate
are for.
Cache invalidation is "never," on purpose. Any URL is paid for at most
once, ever; verification replays from the cached snapshot. That trades
staleness for exact, replayable accounting — the right trade for research
missions that finish in hours. --fresh bypasses the cache per call when
you need today's page.
Workers are sequential by design. One summarize call processes its
URLs one at a time. Parallelism belongs to the tier above: orchestrators
spawn multiple research processes (that's what the 262-worker afternoon
was). Keeping the worker single-threaded keeps it trivially debuggable and
keeps per-worker accounting exact.
Honest user agent. research-bot/0.1 by default, never a fake browser
UA. Bot-walled pages are handled by escalation — Kagi Extract (a licensed,
paid API) then the Wayback Machine — not by impersonation. $RESEARCH_UA
exists if a specific target requires something else; the DuckDuckGo HTML
fallback is best-effort and warns when it stops parsing.
Isn't this LLMs grading LLMs? The verify bench suite has hard ground
truth: hand-planted distortions (plus one organically-wrong case from a real
mission), scored on whether the verifier catches them — no judge
circularity. The summarize/plan suites do use an LLM judge; keep the
judge fixed while comparing worker models, and treat those scores as
relative, not absolute. Suites are small and grow with every real mission
(see bench/README.md for adding cases).
License
MIT — see LICENSE.
Repo post · repo created 2026-07-19, posted 2026-08-06. Every claim on this page cites the repo's own README, so for once the verifier's job was easy.