You've built a RAG pipeline or fine-tuned a model on Databricks, and it produces answers. The harder question is whether those answers are any good, or just good enough to survive a demo in front of people who already want the project to work.
That's where evaluation earns its keep. Databricks paired with MLflow gives you real machinery for scoring an LLM's output: built-in metrics for correctness and groundedness, a framework for writing your own scorers, and a tracking server that remembers every run so you can see exactly when quality slipped. What it doesn't hand you on a plate is which metrics matter for your use case, how to wire them up properly, or where teams usually trip themselves up. That's the part we get asked about most at Shipshape Data, because a large chunk of our work is moving AI projects from a working prototype to something a business actually runs on.
This is a walk through the metrics, the MLflow mechanics, and the mistakes, written for anyone evaluating an LLM or RAG system on Databricks who wants more certainty than "it read fine when I tried it".
What evaluation on Databricks actually covers
Evaluation isn't one test you run once. It's several layers stacked on top of each other, and most of the trouble we see starts when a team only builds one of them. Get the full picture first, then decide which layers your project actually needs.
The quality of the model's own output
Start with the plainest layer: the text the model actually returns. Three things worth measuring here. Correctness: does the answer match a known ground truth, where you have one. Relevance: does it address the question that was asked, rather than a nearby question the model found more interesting. Fluency: does it read coherently, without the stitched-together feel that shows up when a model is straining past its context window.
These three apply whether you're running a single large language model against a prompt or a five-stage pipeline, and they're the floor of any serious evaluation plan, not the ceiling. MLflow's mlflow.evaluate() handles this directly: point it at a dataset of inputs and expected outputs, and its built-in scorers assess responses across these dimensions without you writing the scoring logic yourself. Where the built-in metrics don't fit, you register your own Python function and it slots into the same pipeline, no separate infrastructure required.
Evaluating a RAG pipeline as two separate problems
Once retrieval-augmented generation enters the picture, one evaluation question splits into two. First: did the system pull back the right chunks of context. Second: did the model actually use that context correctly, or did it wander off and invent something plausible-sounding instead.
Those two questions need two different metrics, and conflating them is the single most common mistake we see in RAG evaluation work. A model can sound confident and well-formed while it's quietly working from the wrong source document, and if you only score the final answer, that failure hides in plain sight.
MLflow and Databricks give you dedicated metrics for both layers. Chunk relevance scores whether the documents your retriever surfaced actually relate to the query. Faithfulness scores whether the generated answer stays grounded in that retrieved context rather than drifting into invention. Run both, and you get a clear signal on where a failure originates, retrieval or generation, so you fix the actual broken part instead of guessing and re-prompting.
Writing your own scoring functions
Not every quality dimension has an off-the-shelf metric. If your use case involves a specific domain, a house tone, or a compliance rule your legal team wrote in a document nobody else has read, you need a custom scorer that reflects that. MLflow lets you register a plain Python function as a metric and run it inside the same evaluation call as the built-in scorers, sitting right alongside faithfulness and toxicity in the results table.
This is the part of the platform that earns its keep long-term. You're not stuck with a fixed menu of dimensions someone else decided mattered. You decide what "good" looks like for your application, write it as a function, and apply it the same way on every single run from then on. The result is an evaluation framework that reflects your business, not a generic benchmark built for a use case that isn't yours.
Why this matters once the system is live
Moving a model from a notebook into a live product changes the risk profile completely. In a notebook you control the inputs, you pick the examples, you show people the good runs. In production your model meets whatever a real user types at 11pm on a Tuesday, plus every edge case nobody thought to test for, plus data that quietly drifts underneath it over the following six months. Without a formal evaluation process sitting between you and that reality, you have no way of knowing when outputs degrade, when retrieval quietly breaks, or when the model starts producing answers that are wrong with total confidence.
What skipping it costs you
Teams that skip evaluation during development tend to discover the problem at the worst possible moment: after launch. Hallucinated answers, a retrieval chain that's silently returning stale chunks, responses that drift off-topic. These aren't just bugs to patch. They cost you trust, and trust doesn't come back at the same rate it left. A user who got a confidently wrong answer from your AI assistant once will double-check everything it tells them from then on, which defeats most of the point of building it.
Fixing a quality problem after it's live costs a lot more than catching it in development would have, in engineering time, in support tickets, in the slower and harder job of winning users back. Evaluation is what moves that cost to the left, into the part of the project where a bug is still cheap.
Why one evaluation pass isn't enough
A strong evaluation before launch is necessary. It is not sufficient. Model behaviour shifts as the underlying data your system retrieves against changes, as users start asking questions you didn't anticipate, and as the base model itself gets updated by the vendor without asking your permission first. Wire evaluation into your MLflow workflow so it runs on a schedule or fires on deployment events, and you catch a regression the week it happens rather than the week a customer emails you about it.
Treat evaluation as a pre-launch checkbox and you're blind to exactly the kind of slow drift that eats most production AI systems from the inside. Building continuous evaluation in from day one gives you a running signal on quality at every stage of the system's life, not just a snapshot from the day someone approved the launch.
The metrics worth your time
Knowing which metrics to run is where a lot of teams lose weeks. Databricks LLM evaluation exposes a genuinely wide menu of scoring dimensions, and running all of them on every job is both slow and largely wasted effort. Match the metric to the use case, then add more as the system matures and you learn where it actually breaks.
Retrieval and generation, scored apart
For a RAG application, start with faithfulness and answer relevance as your two most important generation metrics. Faithfulness checks whether the response stays grounded in the retrieved context, penalising anything the model has invented that isn't actually in the source material. Answer relevance checks whether the response addresses what the user asked, which catches the surprisingly common case of an answer that's accurate but pointed at the wrong question.
On the retrieval side, chunk relevance and context precision tell you whether the retriever itself is doing its job. Chunk relevance scores each retrieved document against the query individually. Context precision measures how much of what got retrieved was actually useful, rather than padding the context window with near-misses. Put the two sets together and you get a clean read on whether a failure sits upstream in retrieval or downstream in generation, which tells your engineers exactly where to spend the afternoon.
Toxicity, safety, and the metric that's actually yours
Toxicity scoring and safety classification are not optional if your system talks to end users. MLflow ships built-in toxicity metrics that flag harmful, offensive or off-brand output before it reaches production, and they run against every response in the evaluation set automatically, giving you a systematic floor rather than a manual spot check someone does on a Friday afternoon.
Beyond safety, define at least one metric specific to your own domain early, not as an afterthought once everything else is working. A financial application should score for numerical accuracy. A legal one should score for citation correctness, checking that the case or clause cited actually exists and says what the model claims it says. These domain metrics catch exactly the failures a generic benchmark walks straight past, and encoding them as reusable MLflow scorers means every model version gets checked against the same bar, not whatever the reviewer happened to remember to look for that week.
Setting up evaluation with MLflow in Databricks
mlflow.evaluate() is the entry point for structured evaluation inside Databricks. Give it a model, a dataset of inputs and expected outputs, and a list of scorers, and it hands back a detailed results object plus metrics logged straight into the MLflow tracking server. The whole thing runs inside a Databricks notebook or job, so it sits in your existing workflow rather than demanding a separate evaluation stack to maintain.
What the call actually needs
Start by defining an evaluation dataset as a pandas DataFrame: your input queries, ground truth responses where you have them, and any retrieved context documents your RAG pipeline produced along the way. Then call mlflow.evaluate(), pointing it at your model, either a logged MLflow model URI or a plain callable Python function, and the scorers you want applied. MLflow handles the scoring, the aggregation and the logging inside the active run without further intervention.
A typical call passes four arguments that matter. data is your DataFrame of inputs and expected outputs. model is a logged MLflow model URI or a Python callable. evaluators is set to "default" to switch on the built-in LLM scorers. extra_metrics is a list of whatever custom scoring functions you've written for your own domain.
Log the evaluation inside the same MLflow run as your model artefacts, not a separate one you have to cross-reference by hand later. That keeps your quality evidence and your model version permanently tied together, and it's the difference between a five-minute audit and a half-day archaeology project when someone asks why a deployment got rolled back.
Reading the results and catching regressions
Once you've run evaluations across a few model versions or prompt changes, the MLflow experiment tracking UI lets you put runs side by side. Plot a metric over time, filter by a score threshold, and you can pinpoint the exact run where a quality dimension dropped. That turns regression detection into something direct: if faithfulness falls between version three and version four, you see it in the UI, not three weeks later in a string of confused support tickets.
For evaluation at scale, wire your evaluation jobs into Databricks Workflows on a schedule that matches your deployment cadence. That gives you a continuous quality signal running in the background, rather than a metric someone remembers to check the day after something's already gone wrong.
Using an LLM as a judge without fooling yourself
An LLM judge, a separate model you deploy purely to score another model's outputs, is a genuinely powerful part of the toolkit. It can assess tone, reasoning quality or policy compliance at a speed no human reviewer will ever match. The risk sits in the word "uncalibrated": trust an unchecked judge's scores and you end up optimising your system for what the judge happens to prefer, which is not the same thing as what your users need, even when the two look similar on a spreadsheet.
Choosing and constraining the judge model
Not every model makes a good judge. You want strong instruction-following and consistent scoring behaviour across varied inputs, and you need to constrain that behaviour tightly rather than trust it to interpret intent. Write the judge prompt with an explicit rubric, defined score ranges, and worked examples of what each score level actually looks like. A vague prompt produces variable scores, and variable scores are a metric you can't trust from one run to the next, which defeats the point of measuring anything.
Never let a model judge its own output. It's the one shortcut that quietly poisons every score that comes after it.
Keep the judge model separate from the model it's judging. Use the same model to score its own answers and you build a feedback loop where the judge favours its own style of phrasing, inflating scores without any of them reflecting real quality. Treat the judge prompt itself as a versioned artefact, tested alongside your production model, because a change to the judge prompt moves your evaluation results just as much as a change to the model does.
Checking the judge against actual people
Before you rely on an LLM judge at any real scale, build a small human-labelled validation set covering a representative slice of the inputs you expect. Run the judge against it and measure agreement with the human labels. Good correlation on that sample gives you a reasonable basis for trusting the judge more broadly. Weak agreement means the rubric needs work before you go any further, not more data thrown at the same broken prompt.
Re-validate periodically rather than once and forget it. A judge calibrated against one slice of queries can quietly degrade as your user base shifts or your retrieval corpus changes underneath it, and you won't notice unless you check.
Where we see this go wrong in practice
A few patterns turn up often enough on client projects that they're worth naming directly. Teams build an evaluation dataset once, early, from a handful of examples someone wrote in an afternoon, and never touch it again. Six months later it no longer resembles what real users actually ask, and every score it produces is measuring a system that stopped existing. Refresh the dataset as the product changes, not just when something breaks.
Another one: chasing every available metric because the platform offers it, rather than the two or three that map to an actual failure mode you've seen. A dashboard with fifteen metrics and no agreed threshold for any of them gets ignored within a month. Pick fewer metrics, agree what "good enough to ship" means for each one, and actually act on the number when it moves.
And the quiet one: no baseline. Teams evaluate a new prompt or a new model version in isolation, see a faithfulness score of 0.82, and have no idea whether that's an improvement or a regression because nobody logged what the previous version scored. Every evaluation run should sit next to the run it's replacing. Without that, a metric is just a number, not a signal.
A plan you'll actually keep running
Databricks LLM evaluation isn't a task you finish once and move past. It runs alongside every stage of the system's life, from the first prototype through to whatever it looks like a year into production. Start by picking a small core metric set that matches your use case, build a ground truth dataset from real user queries rather than invented ones, and wire mlflow.evaluate() into your workflow before anything ships, not after.
From there, add retrieval metrics alongside generation metrics, validate any LLM judge against human labels before you trust it, and schedule evaluation runs so a regression shows up in a dashboard instead of a support queue. None of this needs to be elaborate on day one. It needs to run consistently, and it needs someone accountable for reading the results.
If you'd rather have a second pair of eyes on your evaluation setup before you commit a quarter's roadmap to it, talk to us. We spend most weeks in exactly this part of the stack, and we're happy to look at what you've built.