LLM evaluation
Nicholas Hopper has used LLM evaluation in his work at APX Stream and Memector.
Where Nicholas Hopper has used LLM evaluation
Built the evaluation suite that gates every prompt and model change on scored regression sets.
Wrote the specifications and review standards for prompts, evals and agent code, and advises leadership on build versus buy, model choice and cost.
Added an LLM-as-judge entailment check on drafted answers that fails closed when the judge is unavailable or its output cannot be parsed.
Built golden-set benches for document extraction tools, scored on recall, precision and exact-answer accuracy.
Measured recall@1 of 94% and recall@5 of 100% on 50 vague queries that do not contain the template's name, with median latency around 100 ms.