ronitroy

~/projects/doctor-ai-llm-evaluation

LLM evaluation suite for Doctor AI

A RAG-based evaluation framework that tests Ultrahuman's Doctor AI feature and catches regressions, hallucinations and drops in answer quality.

  • LLM evaluation
  • RAG
  • AI testing

Doctor AI is one of Ultrahuman’s AI features, built on a large language model. I designed and built the test suite that evaluates its answers.

What I did

  • Designed and built a RAG-based evaluation framework for Doctor AI.
  • Built custom eval pipelines that detect regressions, hallucinations and output-quality degradations in the model’s responses.

Where

Ultrahuman, as Lead Software Development Engineer in Testing (July 2024 to July 2026).

<- All projects