Side project · Live
Tonight
A two-person recommendation engine where ranking is a typed judgment with a calibrated probability, not a parsed paragraph.
how measured
Hand-labeled scenarios run against the live judgment pipeline in one batched call: 9 scoring cases and 7 conflict-detection cases. I wrote them, and they are deliberately unambiguous, so this is a regression check, not a hard benchmark.
how measured
Across the 7 conflict-detection scenarios (for example a shellfish allergy against a seafood restaurant). Small n, so a floor on calibration rather than evidence of robustness.
What it is
A recommendation engine for two people that answers "what should we watch?" and "where should we eat?". I built it to show AI systems depth, not an LLM wrapper around a prompt.
The approach
Ranking, filtering and tie-breaking are treated as typed judgments. A small purpose-built judgment model scores each candidate per person, hard-filters explicit dislikes and restrictions, and breaks ties on the shortlist. Each answer comes back as a score with a probability distribution and a confidence value, so nothing about why one option beat another is buried in generated prose.
Candidates come from live TMDB and Google Places data. Nothing runs against mocked data.
Key decision and tradeoff
Two models, two jobs. The judgment model never writes text, and an open-weight model served through Groq only ever writes the "why this pick" blurb. Judgment and generation never share a call. That keeps every ranking decision traceable and keeps ongoing cost near zero, at the price of running two integrations instead of one.
Policy lives in plain code. "We both have to like it" is the minimum of two scores, and "weigh both tastes" is the mean. Changing the policy is a code diff, not a re-run of inference.
The eval
An eval script scores 16 hand-labeled scenarios against the live pipeline in a single batched call and reports accuracy plus precision and recall. I'm upfront about its limits: I wrote the scenarios, they are unambiguous by design, and a perfect score on your own eval is a floor, not a ceiling. A harder set with subtle genre overlaps and mixed signals is the natural next step.
What broke
The originally planned generation model was deprecated, and its replacement spends hidden reasoning tokens before it emits visible text. At a tight token budget, the blurb generator silently truncated to a single word with no error. The fix was to lower the reasoning effort and leave enough token headroom for both the hidden and visible parts.
Status
Live. Full source, screenshots and setup are in the repo.