Search Tool Cli

Documentation for Search Tool Cli

Search Tool (C++ CLI)

The Search Tool is a small, dependency-free C++17 CLI for building compact local indexes that agents can query without reading huge datasets into context. It is the search layer of the Dark Mesh, engineered for determinism and speed.

What it does

The Search Tool solves one problem: agents should never page through thousands of records directly.

Instead of an agent loading a full collection into context, the tool:

  1. Syncs records from The Mind (or raw JSONL/files) into a compact local index.
  2. Searches the index with strict result limits and returns trimmed titles, IDs, slugs, and excerpts.
  3. Gets a single record only after search identifies it.

This keeps output small and avoids token burn — the same economy that justifies every tool in the Dark Mesh.

Command surface

search-tool index-jsonl <records.jsonl> <index-file>
search-tool index-files <root> <index-file>
search-tool search <index-file> <query> [--limit N] [--json]
search-tool get <index-file> <collection> <id> [--json]
search-tool stats <index-file> [--json]
search-tool sync-directus <config.json> <index-file>

sync-directus connects to The Mind, pages through configured collections, normalizes records, and writes a local index. Auth comes from /root/.directus.env or environment variables — tokens are never written to the index.

How it speeds up the system

Native C++ means zero interpreter overhead, no framework startup cost, and no dependency tree. The index file format (STIDX001) is a compact binary — one load, fast scans.

Measured on a live index of 269 records straight from The Mind:

  • Full search: ~23ms (query → ranked JSON results)
  • Index load: the same process that serves get and stats
  • Deterministic scoring: weighted substring/token match (title 60 / slug 40 / path 35 / body 15) — no hallucination-prone embeddings

Benchmark

$ time search-tool search the-mind.idx "blog manager agent" --limit 5 --json
real    0m0.023s
user    0m0.013s
sys     0m0.008s

269 records, 23 milliseconds. That is the Dark Mesh retrieval muscle.

Agent code integration

Python (subprocess)

import subprocess, json

SEARCH_TOOL = "/root/agent_dev/search-tool/build/search-tool"
INDEX = "/root/agent_dev/search-tool/the-mind.idx"

def search_mind(query: str, limit: int = 10) -> dict:
    """Run a Search Tool query against the Mind index."""
    cmd = [SEARCH_TOOL, "search", INDEX, query, "--limit", str(limit), "--json"]
    result = subprocess.run(cmd, capture_output=True, text=True)
    result.check_returncode()
    return json.loads(result.stdout)

Python (full agent idiom)

import subprocess, json

TOOL = "/root/agent_dev/search-tool/build/search-tool"
INDEX = "/root/agent_dev/search-tool/the-mind.idx"

def search(q: str, limit: int = 10):
    return json.loads(subprocess.check_output(
        f'{TOOL} search "{q}" --limit {limit} --json'.split()
    ))

def get_record(collection: str, record_id: str):
    return json.loads(subprocess.check_output(
        f'{TOOL} get {INDEX} {collection} {record_id} --json'.split()
    ))

def agent_search_flow(query: str):
    # 1. Search with a small limit — titles, IDs, excerpts only
    hits = search(query, limit=10)
    # 2. Inspect compact results
    for r in hits["results"]:
        print(r["score"], r["collection"], r["title"])
    # 3. Fetch one record only when needed
    if hits["results"]:
        top = hits["results"][0]
        return get_record(top["collection"], top["id"])