A pilot works in the demo. Then someone tries to put it in front of a real customer, and it falls over, because nobody quite trusts the numbers underneath it. That's almost never an algorithm problem. It's a data quality problem, and it's usually the last thing anyone checks.
How to improve data quality starts with an honest assessment of where your data lives, who owns it, and which gaps are costing you, before you spend on any tool. From there, name owners, standardise definitions, validate at the point of entry, cleanse what matters most, fix the architecture underneath, and keep monitoring so gains don't quietly reverse.
We see this pattern constantly at Shipshape Data: duplicate customer records, three different date formats sitting in the same warehouse, a "status" field that means five different things depending on which team filled it in. None of that shows up in a slide about your AI strategy. It shows up six weeks into a project, when someone finally queries the underlying tables and the model's confident answer turns out to rest on a foundation nobody actually checked.
Bad data is expensive in ways that rarely get costed properly. Engineers spend hours reconciling numbers that should already agree. Analysts build the same report three times because three teams don't trust each other's version. And once a model gives one visibly wrong answer, people stop trusting the rest of its output too, and that trust is slow to earn back.
This is a practical guide to fixing it: seven strategies we use with clients, in roughly the order that works. Assess first, then govern, then standardise, then validate at the point of entry, then clean what matters, then fix the architecture underneath, then keep watching so it doesn't slide back. None of these steps requires buying a platform before you understand your problem, which is, frankly, the mistake we see most often.
1. Run an honest assessment before you spend anything
You can't fix what you haven't measured, and most organisations have never measured their data quality. They have opinions about it. An assessment replaces those opinions with a picture of where your data lives, how accurate it is, who is responsible for it, and which problems are genuinely blocking the outcomes you care about.
Map what you have before you touch anything
Start by cataloguing every data source feeding your analytics and AI work: the obvious warehouse tables, but also the spreadsheet a finance analyst maintains by hand, the CRM export nobody remembers setting up, the API feed that has quietly been failing for months. Note where each source lives, what format it is in, how often it updates, and who has access. This sounds tedious because it's tedious. It's also the only way to see the whole picture rather than the corner of it your team happens to work in.
Once you have the map, profile the data itself. Look for missing values, duplicate records, outliers that don't make sense, and schema mismatches between systems that are supposed to describe the same thing. Check specific dimensions: is a customer record complete, is it consistent across systems, is it accurate against a source of truth, is it timely enough to be useful. This baseline is what tells you how to improve data quality in your specific environment, rather than applying a generic checklist that misses your actual problems.
Rank the gaps that matter
Not every quality issue deserves the same attention, and treating them as if they do is how improvement projects stall. Score each gap on two things: the business impact if it stays broken, and the cost of fixing it. Datasets feeding customer-facing AI, regulatory reporting, or revenue decisions go to the top of the list. A slightly inconsistent internal spreadsheet that three people glance at once a month doesn't.
We have watched teams spend months polishing a dataset almost nobody uses, while the one feeding their flagship AI feature sat untouched. Prioritising by impact, not by what is easiest to fix, is what gets you visible results in the first few months instead of a tidy but invisible improvement three years from now.
2. Put someone in charge of the data
Data quality fails quietly when nobody owns it. Not dramatically, not all at once. A field starts getting filled in inconsistently, a validation rule gets skipped under deadline pressure, a definition drifts slightly between two teams, and eighteen months later nobody can explain why the numbers don't add up. Governance is what stops that drift, by assigning clear responsibility for accuracy, consistency, and fitness for purpose.
Name owners and stewards
Identify specific people, not teams, who own the critical data domains: customer records, product hierarchies, financial transactions, whatever matters most to your business. Owners make the calls on definitions, quality standards, and access. Below them, appoint stewards who handle the day to day: monitoring quality, resolving exceptions, being the person a confused analyst can message. "The data team owns it" isn't ownership. A name is ownership.
Data quality isn't a job description. Somebody specific has to be able to say, this is mine, when a number looks wrong.
Decide who decides
Write down who has the final say when two teams define "active customer" differently, or disagree about what a quality threshold should be. Set policies for retention, privacy, and compliance, and exception handling. Build an approval path for schema changes and new data sources, so nobody discovers a breaking change after it has already broken three downstream reports. Clear decision rights are what stop governance turning into an endless negotiation everyone quietly avoids.
Make governance part of the job, not a meeting about the job
The fastest way to kill a governance programme is to bolt it onto a monthly meeting people attend without changing what they do the other twenty-nine days. Put quality checks and approval gates inside the pipelines and interfaces people already use: a required field in the entry form, an automated check in the deployment pipeline, an alert that fires before a bad load reaches production. Governance that lives inside the workflow survives. Governance that lives in a slide deck gets forgotten by the second quarter.
3. Agree on what your words mean
Inconsistent formats and definitions sabotage analytics before anyone opens a dashboard. If sales calls a customer anyone who has ever filled in a form, and finance calls a customer someone who has paid an invoice, every report that joins the two datasets is quietly wrong from the start. Standardisation gives your organisation one language. Documentation makes sure that language survives when the person who defined it moves on.
Write the definitions down
Define what each critical term means: what counts as an active customer, how revenue gets calculated, what makes an order complete. Write it in plain language two departments can both read without an interpreter. When two teams already use the same word differently, don't quietly pick a winner. Either agree a single authoritative definition, or split them into two clearly named attributes. The point is removing the ambiguity that lets two people pull the "same" report and get different numbers.
Line up formats, codes, and reference data
Standardise date formats, currency codes, country names, and units of measurement across every system that touches them. Build a master list of allowed values for anything categorical: product types, customer segments, status codes. Once teams pull from the same reference list rather than inventing new variants of "Active", "ACTIVE" and "actv", automated checks can flag entries that fall outside the standard, instead of drowning in false positives from formatting noise alone.
Keep a catalogue people use
Build a searchable record of what data exists, where it lives, who owns it, and how to get access. Document lineage too, so people can see which systems feed a given dataset and what happens to it along the way. A catalogue that goes stale the week after launch is worse than no catalogue, because people keep trusting it after it stops being true. Update it whenever a source, definition, or integration changes, and treat that update as part of the change rather than an afterthought nobody gets round to.
4. Stop bad data at the door
Prevention beats correction every time. Catching an error while someone is still filling in a form, or the moment a record lands in your pipeline, costs a validation rule and a clear message telling them what to fix. Catching the same error six months later, once it has spread through four downstream systems, costs a data steward the better part of a week. This is the strategy that moves how you improve data quality from firefighting after the fact to catching problems at the source.
Build validation close to where data is created
Put validation directly in the entry forms, APIs and upload interfaces where people and systems first submit information: required fields, sensible value ranges, format patterns, referential checks against records that should already exist. Reject bad entries immediately, with a message that tells the person what went wrong rather than a generic error code. Catching a mistake while the context is still fresh in someone's head is far cheaper than reconstructing that context during an investigation weeks later.
Automate the checks inside your pipelines
Every data pipeline, ETL job and integration should carry its own automated checks: schema compliance, nulls in fields that can't be null, statistical outliers, logical consistency between related fields, an order can't ship before it was placed, for instance. When a check fails, stop the pipeline rather than letting questionable data quietly reach production. It feels aggressive the first time a build breaks over what looks like a minor issue. It's considerably less aggressive than explaining to a client why their dashboard has been wrong for three weeks.
Give exceptions somewhere to go
Not everything that fails validation should be rejected outright. Some of it is a legitimate edge case your rules haven't caught up with yet. Route failures to a steward who can investigate, correct what needs correcting, and update the rule if the exception turns out to be normal rather than an error. Log every failure with enough context to troubleshoot it, and review the pattern of failures periodically. If the same field keeps failing the same check, that stops being a data problem. It's a process problem upstream.
5. Cleanse, deduplicate, and enrich the data that matters
Raw data rarely arrives fit for use. Cleansing fixes the errors and inconsistencies that quietly undermine model accuracy. Deduplication removes the records that inflate your counts and confuse anything trying to learn a pattern from them. Enrichment adds the context that was missing in the first place. None of this needs to happen everywhere at once, and trying to make every dataset perfect is a good way to run out of budget before you fix the ones that matter.
Profile before you clean anything
Prioritise the datasets that directly support revenue, compliance, or the AI initiatives leadership actually cares about. Profile them for real numbers: what proportion of records are duplicated, how complete are the key fields, how consistent are the formats. Score issues by their effect on the business rather than how technically severe they look on paper. This is what stops cleansing turning into an open-ended project that never quite finishes, because there is always one more field that could be tidier.
Deduplicate, then fix what's obviously wrong
Match duplicate records across several attributes rather than trusting a single ID field, which will always miss the customer who signed up twice with a typo in their email address. When automated confidence in a match is low, put it in front of a person rather than merging blind. A wrong merge is often worse than a duplicate you leave alone. Fix the obviously systematic errors, formatting inconsistencies, invalid codes, values outside a sane range, with automated rules once you understand the pattern. And write down what you changed. A colleague who spots an unexpected transformation six months later should be able to find out why it happened, not have to guess.
Enrichment turns clean data into useful data
Once a dataset is accurate, add the context that makes it valuable: geographic codes, industry classifications, calculated fields like customer tenure or transaction velocity, links between customer records and the transaction history that explains their behaviour. This is the difference between data that's merely correct and data that tells you something. Cleansing removes what's wrong. Enrichment adds what was missing.
6. Break down the silos underneath everything
Isolated systems that can't talk to each other are where most of this starts. Departments build their own tools, those tools can't exchange data cleanly, and you end up with three versions of the same customer sitting in three databases that disagree with each other. Legacy platforms make it worse by locking data into formats modern AI tooling can't reach without a translation layer nobody wants to build twice. Fixing quality problems on top of an architecture like that is a permanent tax. Fixing the architecture removes the tax.
Consolidate onto platforms people trust
Move critical datasets out of fragmented systems and into a centralised platform, a cloud warehouse or data lake that supports one consistent way of accessing data rather than five slightly different ones. Build integration layers that standardise formats as data moves between systems, instead of leaving every downstream team to handle the inconsistencies themselves. Where it makes sense, put in place a single authoritative source for core entities like customers and products, so "which version is correct" stops being a weekly argument.
Make lineage visible so root causes are findable
Track where data originates, what transformations happen to it, and which systems consume the result. When something breaks downstream, this is what lets you trace it back to the actual cause in minutes instead of days. Visibility here does more than speed up debugging: it lets you assess the blast radius of a proposed change before you make it, rather than finding out afterwards which five reports you just broke.
Design for the workload you have
Build platforms that can carry real analytics and model training volumes without slowing the transactional systems underneath them. Give data scientists a way to version and experiment with data safely, away from production. Choose architecture that can grow with your data volumes and with however complex the AI work becomes, because whatever you build now will be under more pressure in two years, not less.
7. Keep watching after the cleanup
Quality improvement that stops the day the project ends doesn't stay improved. Source systems change, integration logic drifts, new data patterns turn up that your original rules never anticipated. Continuous monitoring protects the investment you already made in the six strategies above, instead of watching it quietly decay over the following year.
Pick metrics that tie back to outcomes
Set measurable indicators for each critical domain: completeness percentage, duplicate rate, how quickly updates land. Then connect those numbers to something a business person cares about: model accuracy, report reliability, how confident people feel making a decision off the data. Track both the current level and the trend, because a metric that's stable but mediocre needs a different conversation to one that's dropping fast.
Put it on a dashboard someone looks at
Build dashboards that show current quality against defined thresholds, with history so people can see whether things are getting better or worse. Set automated alerts so a steward hears about a quality drop the day it happens, not the day a customer complains. Fold quality monitoring into whatever observability tooling already gets attention in your organisation, so a data problem gets treated with the same urgency as a system outage, rather than sitting in a queue nobody checks.
Close the loop with the people using the data
Ask analysts and data scientists directly what quality problems they run into. Not a survey once a year, an actual conversation. When something goes wrong, look for the root cause instead of patching the symptom for the third time. And when quality does improve, tell the people who depend on that data. Trust, once lost, takes longer to earn back than the original fix took to build.
Where to start
Seven strategies, and no shortcut through them. Assess first, so you are fixing the right things. Assign ownership, so improvements don't quietly reverse the moment nobody is watching. Standardise definitions, so two departments stop arguing about numbers that should already agree. Validate at the point of entry, so bad data never gets in cheaply enough to be worth catching later. Cleanse and enrich what actually matters rather than everything. Fix the architecture underneath, so quality improvements have somewhere stable to live. Then keep watching, because the work doesn't finish. It just gets quieter.
We built Shipshape Data because we kept running into the same wall with clients: an AI project would stall, and the actual problem sat three layers below the model, in data nobody had ever properly assessed. If any of the seven steps above feel like the gap in your organisation, that is exactly where we start. Talk to us, and we will help you work out which of these seven matters most right now.
Frequently asked questions
How do you improve the quality of data?
Improving data quality starts with an honest assessment: map every source feeding your work, profile it for missing values and duplicates, and rank the gaps by business impact rather than by whichever is easiest to fix. From there, name an owner for each critical dataset, standardise the definitions two teams keep disagreeing over, and validate data at the point it's created rather than cleaning it up months later.
How do you improve data quality across an organisation?
Across an organisation, data quality fails quietly when nobody owns it, so the first fix is naming specific people, not teams, as owners of each critical domain. Pair that with a catalogue people keep properly updated, validation built into the forms and pipelines already in daily use, and a dashboard that shows quality trends to someone who will act on a falling number.
How do you improve data quality in a data warehouse?
In a data warehouse, quality problems usually trace back to systems that can't talk to each other, so consolidating critical datasets onto one platform with a single authoritative source for core entities like customers and products removes a big source of the mess. Pair that consolidation with visible lineage, so when something breaks downstream you can trace it back to its actual cause in minutes rather than days.
How do you improve data quality in healthcare settings?
In healthcare and any setting with duplicated or long-lived patient records, the same fundamentals apply with less room for error: assess where records really live, assign a named owner for critical domains, and validate entries at the point of creation so a duplicated record never gets the chance to fragment a patient's history across systems. Continuous monitoring matters most here, since a quality problem that goes unnoticed compounds every time that record is used again.
How do you improve data quality in research?
In a research context, the same seven-strategy approach applies: assess what data you have before assuming it's fine, agree clear definitions so two researchers aren't silently measuring different things, validate entries as they are collected rather than after analysis has started, and keep monitoring so quality doesn't quietly decay over the course of a longer study.