akiva

LLM spend and routing analyzer: ingest gateway, trace and usage exports, price requests by token class, and find the cheapest routing that keeps quality.

Contracts for LLM-shaped data: JSON Schema contracts, LLM presets, ODCS interop and drift checks.

LLM safety and compliance evidence: policy tests, garak/promptfoo/PyRIT import, control mapping and CI gates.

Retrieval regression testing in CI: trec_eval-equal metrics, per-query diffs, significance tests and leakage checks.