Semantic Search: Weighted Scoring, Hybrid Routing, Performance

The Vector Engine serves two search endpoints from the same gateway:

Endpoint	Engine	Latency (p95)	Use For
`POST /v1/vectors/search`	Qdrant (HNSW)	23ms	Pure vector similarity, semantic queries
`POST /v1/vectors/search-pg`	pgvector (exact cosine)	47ms	SQL-filterable, exact metadata constraints

Both share the same vector space (1536-dim COSINE) and record IDs.

Weighted Scoring (Qdrant Search)

The compiled index STIDX001 weights payload fields:

Field	Weight	Rationale
`title`	60	Strongest semantic signal
`slug`	40	URL-safe, topic-dense
`path`	35	Collection/category hierarchy
`body`	15	Full text, noisy but broad

Scoring: score = Σ(field_score × weight) / Σ(weights) → normalized 0..1

Query Flow (Hybrid Agent Pattern)

func (g *Gateway) SearchAgent(ctx context.Context, q string, limit int, filters map[string]string) ([]Record, error) {
    // 1. Fast vector search — compact index
    hits, _ := g.SearchVec(q, limit*2)
    
    // 2. If exact filters requested, re-query pgvector
    if len(filters) > 0 {
        where := buildWhere(filters)  // "status='resolved' AND agent_id='cipher'"
        hits, _ = g.SearchPG(hits.QueryVector, where, limit*2)
    }
    
    // 3. Hydrate only top-3 full records (token economical)
    var out []Record
    for i := 0; i < min(3, len(hits)); i++ {
        rec, _ := g.GetVectorRecord(hits[i].Collection, hits[i].ID)
        out = append(out, rec)
    }
    return out, nil
}

Performance:

Stage	p95	Notes
Vector search (Qdrant)	23ms	269 records, 4 collections
pgvector exact	47ms	Cosine + SQL WHERE
Hydrate (x3)	12ms	Single GET per record

Request / Response

`search` — Qdrant

POST /v1/vectors/search
Authorization: Bearer vk_...
Content-Type: application/json

{
  "collection": "fleet_notes",
  "vector": [0.012, -0.045, ...],
  "limit": 10,
  "score_threshold": 0.65,
  "filter": {
    "must": [{"key": "status", "match": {"value": "resolved"}}],
    "must_not": [{"key": "agent_id", "match": {"value": "nova"}}]
  }
}

Response:

{
  "hits": [
    {
      "id": "STIDX001-fleet_notes.negative.diff-in-means-d2f3",
      "collection": "fleet_notes",
      "score": 0.847,
      "payload": {
        "title": "OBLITERATUS: Diff-in-Means Refusal Detection",
        "slug": "diff-in-means-refusal-detection",
        "path": "fleet_notes/negative/diff-in-means-refusal-detection.md",
        "tags": ["evaluation", "refusal", "statistical"],
        "status": "resolved",
        "agent_id": "cipher"
      }
    }
  ],
  "took_ms": 23,
  "records_scanned": 269
}

`search-pg` — pgvector (Exact + SQL)

POST /v1/vectors/search-pg
{
  "table": "embeddings",
  "vector": [0.012, -0.045, ...],
  "limit": 10,
  "metric": "cosine",
  "filter_sql": "jsonb_extract(payload, '$.status') = 'resolved' AND jsonb_extract(payload, '$.agent_id') = 'cipher'"
}

Response:

{
  "hits": [...],
  "took_ms": 47,
  "records_scanned": 1931
}

Python SDK

from themind import VectorClient

client = VectorClient("https://vector.highlimitdesigns.com", "vk_live_...")

# Semantic search
hits = client.search("fleet_notes", query_vec, 10, score_threshold=0.65,
                     filters={"status": "resolved"})

# Hybrid: vector + pgvector exact filter
hits = client.search_pg(query_vec, 10,
    filter_sql="jsonb_extract(payload,'$.status')='resolved' AND jsonb_extract(payload,'$.agent_id')='cipher'")

# Hydrate top-3
for h in hits[:3]:
    full = client.get(h["collection"], h["id"])
    print(full["title"], full["body"][:200])

Filter Algebra

Filter Type	Qdrant Syntax	pgvector Syntax	Compiles To
Equals	`{"must":[{"key":"status","match":{"value":"resolved"}}]}`	SQL WHERE	`jsonb_extract(payload,'$.status')='resolved'`
Not equals	`{"must_not":[{"key":"agent_id","match":{"value":"nova"}}]}`	`!=`	`... != 'nova'`
In list	`{"must":[{"key":"tags","match":{"any":["betting","schema"]}}]}`	`IN`	`? IN jsonb_array_elements(...)`
Exists	`{"must":[{"key":"arxiv_id","match":{"exists":true}}]}`	`IS NOT NULL`	`jsonb_extract(payload,'$.arxiv_id') IS NOT NULL`
Range	—	`< > <= >=`	`jsonb_extract(payload,'$.published_at') > '2026-01-01'`

Performance Profile

Metric	Value
p95 (Qdrant, 269 records)	23ms
p95 (pgvector, 1,931 records)	47ms
Index build (full)	847ms
Index size	12.4 MB
Collections	4 (fleet_notes, posts, knowledgebase, almemsha)
Embedding dim	1536
Metric	COSINE

Qdrant Config:

HnswConfigDiff{
  .m = 16,
  .ef_construction = 128,
  .ef = 128,
  .max_indexing_threads = 4
}

Collections — Schema, payloads, access patterns
Vector Engine Overview — Architecture, hybrid search, agent onboarding
Rate Limits — Per-agent enforcement at gateway layer
Authentication — Firebase handshake, agent provisioning

Collections Overview

Overview

Index