A model returns a confident answer that is simply wrong. A dashboard shows two numbers for the same metric depending on who built the query. An engineer spends an afternoon chasing a broken pipeline, fixes it, and watches the same fault reappear three weeks later under a different name. None of these are model problems or dashboard problems. They are data quality problems wearing a disguise, and the fix starts further upstream than most teams look.
Soda is one of the more sensible answers to that problem. It is a testing framework that checks your data the way a test suite checks your code: you write rules about what a table should look like, Soda runs them against the real thing, and it tells you the moment reality stops matching the rule. You can run it as a free command-line tool, or pay for the hosted platform that adds monitoring, alerting and a dashboard on top. We work with data quality tooling constantly at Shipshape Data, and Soda is one of the tools we reach for when a client's checks live nowhere and nobody trusts the numbers.
This guide covers what Soda actually does, how to plan an implementation that does not collapse under its own alert volume, how to install and configure it against a real warehouse, how to wire it into CI/CD and orchestration so checks run without anyone remembering to click a button, and how it stacks up against Great Expectations and the anomaly-detection platforms. Read it end to end if you are starting from nothing, or jump to the comparison section if you already know Soda does what you need and just want to know if it is the right pick.
What Soda data quality actually does
Soda validates datasets against rules you write in YAML, using a syntax called SodaCL, short for Soda Checks Language. A check might say a table has more than a thousand rows, that no email field is blank, or that an order ID never repeats. Soda runs the check against Snowflake, BigQuery, Redshift, Databricks, PostgreSQL or any of its other supported connectors, and returns a pass or a fail. Fail a check and you find out immediately, rather than three reports downstream when someone notices the totals look strange.
SodaCL ships with more than twenty-five built-in metrics covering freshness, completeness, validity and distribution, which covers most of what a data team needs without writing custom SQL. When the built-ins fall short, you drop into a raw SQL check and Soda treats the result the same way. The language reads close to plain English, which matters more than it sounds: a check that an analyst can read and correct without asking an engineer gets maintained. A check buried in a wall of Python does not.
Soda Core: the free, open engine
Soda Core is the command-line tool and Python library underneath everything. Install it, point it at a data source, run a scan, get a result in your terminal. It is free, it is fully functional, and plenty of teams never touch anything beyond it. If your appetite is version-controlled checks that live next to your dbt models and run in CI, Core does the whole job on its own.
Soda Cloud: monitoring on top
Soda Cloud is the commercial layer: scheduled scans that run without a human triggering them, a dashboard showing quality trends across every dataset you have checks on, and alert routing so a failed check reaches the person who owns that data rather than a channel nobody reads. It also adds anomaly detection that flags patterns nobody explicitly wrote a rule for, which is useful for the failure modes you have not thought of yet. The trade is the one every open-core product makes: Core gives you control, Cloud gives you convenience, and which one your team needs depends on whether you already have somewhere to run scheduled jobs and somewhere to send alerts.
Step 1: plan before you write a single check
The most common way teams waste the first month with Soda is skipping straight to writing checks. They point Soda at every table in the warehouse, generate a wall of green and red results, and by week three nobody reads the Slack channel any more because it fires forty times a day about a table three people care about. Planning is dull compared with writing YAML, but it is the difference between a data quality system people trust and one they mute.
Work out which datasets actually matter
Start with the tables that feed production systems, customer-facing dashboards and anything that triggers an automated decision. These are the datasets that break something visible the moment they go wrong, so they earn checks first. Datasets that several teams depend on internally deserve the same priority even if no customer ever sees them directly, because a quality problem there multiplies across every team downstream rather than staying contained.
A rough three-tier split works for most estates. Tier one covers production systems, revenue figures and anything customer-facing. Tier two covers internal dashboards, operational reporting and day-to-day team workflows. Tier three is everything experimental, archived or rarely queried. Build your first checks against tier one only, get them running reliably, and let the team learn the patterns before you extend coverage down the list. Trying to cover all three tiers on day one is how you end up with checks nobody maintains on data nobody looks at.
Decide what "good" means for each dataset
Every dataset needs its own definition of quality, because a rule that makes sense for a revenue table can be nonsense for a log table. Revenue figures usually need zero missing values and exact decimal precision, full stop. Behavioural logs can tolerate the odd gap but need strict ordering on timestamps, because an event log with entries out of sequence is nearly useless for anything downstream. Write these standards down in plain language before you translate a single one into a check, otherwise you end up encoding assumptions nobody agreed to.
Four questions cover most of what you need to decide per dataset: which fields can never be null, and what percentage of missing values actually breaks something downstream; what format each field must follow, and which values fall outside an acceptable range; how related fields connect to each other, and which cross-field rules genuinely have to hold; and how old the data can get before it stops being useful, and how often you expect it to refresh. Answer those four for a table and you have most of your checklist.
Build a coverage plan with a sane check frequency
Not every check needs to run hourly. Structural checks like row counts and schema shape are cheap, so run them often. Business logic checks cost more to compute and change more slowly, so daily is usually plenty. Expensive distribution checks that scan whole columns for statistical drift belong on a weekly cadence unless you have a specific reason to run them more often. Getting this wrong in either direction costs you: too frequent and you burn warehouse compute on checks that never change; too sparse and a broken pipeline runs for a week before anyone notices.
Step 2: install and configure Soda
Once the plan exists, installing Soda itself takes minutes. You need Python 3.8 or later on the machine that will run scans, and a set of credentials for whichever warehouse you are pointing it at.
Install the right variant for your warehouse
Soda Core ships as a family of platform-specific packages, each bundling the database driver that connector needs. Install the Snowflake variant if that is your warehouse, the BigQuery variant for BigQuery, and so on through Redshift, PostgreSQL and Databricks. You only install the packages for the sources you actually query, and adding a new source later is a case of installing one more package rather than reconfiguring anything that already works. Once it is in, run the version command in your terminal and confirm you get a version number back before moving on.
Point it at your data
Connection details live in a separate configuration file from your checks, which matters because it keeps credentials out of the files you commit and share. That file names the data source, states its type, and lists the connection parameters: host, port, username, password, database and schema for something like PostgreSQL, with account and warehouse names required instead for Snowflake, project and dataset identifiers for BigQuery, and cluster details for Databricks. Reference environment variables for anything sensitive rather than typing a password into the file directly, and if the file ever does end up with a hardcoded credential in it, even for a throwaway development environment, keep it out of version control entirely. That rule sounds obvious until the third person on the team copies a working config file with a real password still in it because it was faster than reading the docs.
Write and run your first scan
Your checks live in a second YAML file, separate from the connection config, and this is where the actual rules go: a row count above zero, no missing customer IDs, an order date no older than a day. Run the scan command with your data source, your configuration file and your checks file, and Soda connects, executes every check, and prints pass or fail results straight to the terminal. A failed check tells you which rule broke and shows the actual value it found against what you expected, so debugging starts from a specific number rather than a vague sense that something looks wrong.
The value of a data quality check is not the green tick. It is how fast the red one tells you exactly what broke.
Step 3: wire Soda into the workflows you already have
A scan you run by hand catches problems after the fact, usually once someone has already noticed the numbers look off. The real value shows up once checks run automatically at the points where bad data would otherwise slip through, which means CI/CD for anything that changes structure and orchestration for anything that moves data.
Put checks in the deployment pipeline
Your CI/CD pipeline is the first natural checkpoint, because it is where schema changes, dbt model edits and transformation logic get reviewed before they ship. Configure the pipeline to run Soda whenever someone touches a schema or a model, and make a failed check block the deployment the same way a failed unit test would. In practice that is a short step in a GitHub Actions workflow (or whichever CI tool you use): install the right Soda package, pull credentials from your secrets store, and run the scan command against the environment you are deploying to. Keep the check files in the same repository as the transformation code they protect, so a change to what "good" means for a table shows up in the same pull request as the code that changes the table, with the same review and the same history.
Add checks to the orchestration layer
Airflow, Prefect, Dagster and similar tools decide when data actually moves, which makes them the natural place to add validation as an explicit step in the DAG rather than an afterthought. Run a check immediately after loading data from an external source, and run another after each transformation step completes, so you catch problems both at the source and in what your own logic produced from it. Configure the orchestrator to halt on a failed check rather than continuing regardless. Letting a pipeline run on with data you already know is bad just moves the debugging further downstream and makes the eventual root cause harder to find, not easier.
Get alerts to the right person, not the loudest channel
A check that fails silently in a log file might as well not exist. Route alerts by who actually owns the data rather than dumping everything into one shared channel that everyone eventually mutes, whether that routing runs through Slack, email, PagerDuty or the Soda Cloud integration directly. A good alert names the check that failed, the dataset it failed on, and the actual value against what was expected, so the person reading it can start fixing rather than starting by reproducing the problem. Not every check deserves the same urgency either. A broken revenue calculation should page someone immediately. A completeness check on a low-stakes internal table can wait for a Slack message reviewed during business hours. Treat every failure as equally urgent and you train the team to ignore all of them within a month.
Step 4: how Soda compares to the alternatives
Soda is not the only option, and picking a data quality tool before you understand your own checks and cadence is how you end up rebuilding the whole thing eighteen months later against a tool that never fit.
Soda versus Great Expectations
Great Expectations is the closest open-source alternative, and the two are more similar than different: both validate data with configuration files, both support the major warehouses, both are free at the core. The real difference is how you write a check. Great Expectations builds checks as expectation objects, which is more structured but also more verbose to read and write. SodaCL reads closer to a sentence. A row count check in Great Expectations takes noticeably more lines than the equivalent in SodaCL, and that gap compounds once a team is maintaining hundreds of checks rather than five.
Where Great Expectations pulls ahead is data profiling: it can generate a starting set of validation rules automatically from an existing dataset, which is a genuinely useful way to bootstrap coverage on a table you have never written a check against before. Soda leans the other way, favouring checks a human writes deliberately, with error messages that tend to be clearer and scans that tend to run faster. If your team wants automatic documentation and profiling out of the box, Great Expectations earns its complexity. If you want checks your whole team, not just the engineers, can read and edit without training, Soda is the simpler choice.
Soda versus the anomaly-detection platforms
Commercial platforms such as Monte Carlo and Datafold take a different approach entirely: instead of you writing rules, they learn what normal looks like from your data's own history and alert you when something drifts from that pattern. That gets you coverage fast, particularly on tables where you would not know what to check for until something already went wrong, but it costs you control over exactly what triggers an alert, and that trade does not suit every team.
Cost shapes this decision as much as capability does. Soda Core stays free indefinitely, with no ceiling tied to data volume or table count. Commercial platforms typically charge on one or both of those, so the bill grows with your estate whether or not your check volume grows with it. Run the maths on your actual data volume before committing to either path, and test your top two candidates against a real pipeline you already find painful rather than a clean demo dataset. A tool that looks flawless in a fifteen-minute demo can still miss the exact transformation that keeps breaking your Friday report.
Getting the sequencing right
None of this works if the underlying data model is a mess before the first check ever runs. We see this pattern often: a team adopts Soda, writes solid checks, and the checks keep failing not because the data is genuinely broken but because nobody agreed what the table was supposed to contain in the first place. Data quality tooling assumes there is a stable target to validate against. If your data architecture itself is still shifting under you, checks will flag every shift as a failure, and the team stops trusting the alerts within weeks.
That is really what data quality management is: not a tool bolted onto the end of a pipeline, but a discipline that starts with agreeing what correct looks like, gets encoded once that agreement exists, and then gets automated so the agreement holds without someone re-checking it by hand every week. Soda is a strong piece of that discipline. It is not a substitute for having the discipline in the first place.
Ownership matters just as much as the checks themselves. A check with no named owner is a check nobody fixes when it fails, and that gap is usually a symptom of wider data governance gaps rather than a Soda configuration problem: nobody has actually agreed who is responsible for a given dataset's quality. Fix that ownership question before you scale checks past your first tier, because a hundred well-written checks routed to nobody in particular deliver the same outcome as no checks at all.
Where to start
Start small. Pick your handful of tier-one datasets, write the checks that would have caught your last three data incidents, and get them running reliably in CI before you touch anything else. Track your coverage honestly as you go, because a dashboard that looks comprehensive but only checks row counts is not actually protecting you from the failures that hurt. Expand outward once the pattern is proven and the team trusts what the alerts tell them.
If you are further back than that, still mapping which datasets matter or untangling a data model that will not hold still long enough to validate, that is the part we spend most of our time on at Shipshape Data. Talk to us and we will help you work out where your foundation needs work before you pick a tool to sit on top of it.