--- title: "Data Quality Issues: Common Causes and How to Fix Them" description: "The data quality issues teams see most, traced to their root causes, with the fixes that hold and why AI turns small issues into visible failures." canonical: https://shipshapedata.com/resources/data-architecture/data-quality-issues/ language: en-GB author: "Emile Van Zyl" author_role: "MarTech Automation Consultant" author_url: https://www.linkedin.com/in/emile-vanzyl-enhancing-your-human-advantage/ date_published: 2026-10-24 date_modified: 2026-10-24 --- # Data quality issues: common causes and how to fix them > The data quality issues teams see most, traced to their root causes, with the fixes that hold and why AI turns small issues into visible failures. Section: Home > Resources > Data & architecture Canonical page: https://shipshapedata.com/resources/data-architecture/data-quality-issues/ By Emile Van Zyl, MarTech Automation Consultant (https://www.linkedin.com/in/emile-vanzyl-enhancing-your-human-advantage/) | published 2026-10-24 A finance report and a sales dashboard show different revenue for the same month. The same customer appears three times in the CRM, each time with a different address. A forecast that looked sound last quarter starts to drift, and nobody can say which input changed. Each of these is a data quality issue, and each one costs time before anyone has fixed it. Data quality issues are flaws in data that stop people and systems from using it with confidence: duplicate records, figures that disagree between systems, stale values, missing fields and joins that fail. Most are symptoms rather than causes. They trace back to missing ownership, unclear definitions and checks that run too late, so the fix sits in the foundation. The usual list of symptoms is familiar: inaccurate, incomplete, duplicate, inconsistent and outdated data. That list is useful, but it stops at the point where the work starts. This guide is for the people who have to fix the problem, usually a head of data, a finance lead or the person handed an AI mandate. It names the issues you will see, traces each one to its root cause, and explains why the fix belongs in the data foundation rather than in another round of clean-up. If you need the basics first, start with our guide to what data quality is and why it matters. ## What counts as a data quality issue A data quality issue is any gap between what the data says and what the business needs it to say. That gap can be a wrong value, a missing value, a value in the wrong format, or a value that was correct last month and is not correct now. It can also be two correct values that disagree because two systems define the same term in different ways. Quality is measured against a purpose. A postcode field that is half empty may not matter to a monthly revenue report. It matters a great deal to a delivery routing model. This is why the standard data quality dimensions exist. Accuracy, completeness, consistency, timeliness, validity and uniqueness give you a shared way to say what is wrong and how badly. The dimensions describe the symptom. They do not tell you the cause. A duplicate record breaks the uniqueness dimension, but the cause might be a missing match rule, a second system that creates its own records, or a sales team with no reason to search before they type. If you fix the symptom and leave the cause, the same issue returns next month. > A data quality issue is usually a symptom. The cause is a decision nobody made about who owns the data and what it means. ## Common data quality problems and their causes The problems below are the ones people notice first, because they show up in reports, in contact with customers and in meetings where two teams bring two different numbers. For each one, we set out the symptom, the root cause we most often find and the fix that holds. ### Duplicate records The symptom is the same customer, supplier or product held more than once, often with small differences in spelling or address. Counts are inflated, contact is repeated and history is split across records. The root cause is rarely careless typing. It is usually that more than one system can create the record, with no agreed key and no rule for which version wins. The fix is a single owner for each core entity, a match rule, and one golden record that other systems read from. That is the discipline of master data management. ### Conflicting figures across systems The symptom is finance reporting one revenue figure and the sales dashboard reporting another. Both teams can defend their number. The root cause is a missing shared definition. One team counts revenue on invoice and the other on order. One excludes credit notes and the other does not. No amount of cleaning fixes this, because both datasets are correct by their own rules. The fix is one agreed definition for each business term, written down and built once in a semantic layer that every report uses. Our guide to data reconciliation covers how to trace the point where two systems part ways. ### Stale records The symptom is data that was true once: a contact who has left the company, a price that has changed, or yesterday's stock levels shown as today's. The root cause is usually an unowned refresh. A feed fails quietly, a manual export stops when the person who ran it changes role, or nobody set a freshness requirement in the first place. The fix is a stated freshness target for each dataset, a check that runs against it, and an owner who is told when it fails. ### Broken joins The symptom is a report that drops rows, counts them twice, or shows blanks where a related value should be. The root cause is keys that do not line up: a customer ID stored as text in one system and as a number in another, a code list that changed upstream, or a field renamed without notice. The fix is an agreement between the team that produces the data and the teams that use it, so changes to schema and meaning are agreed before they ship. Our guide to data contracts explains how this works in practice. ### Missing and invalid values The symptom is blank mandatory fields, dates in the future, negative quantities or free text where a code should be. Human error is part of the cause, but the larger part is design. The system allowed the bad value in, and nobody downstream had stated what a valid value looks like. The fix is validation at the point of entry where possible, and tests at the point of load where it is not. The pattern across all five problems is the same. The visible problem sits in the data. The cause sits in ownership, in definitions and in the absence of a checkpoint where bad data stops. ## How to spot data quality issues early Most organisations find out about a data quality issue when a user complains. By then the bad figure has already appeared in a board pack or in an email to a customer. The aim is to find the issue before your users do. These practices make that possible. - Profile the data you rely on. Run counts, null rates, distinct values and ranges on the tables behind your most used reports. Unusual results point you at the problems worth investigating. - Write down what good looks like. For each important dataset, state the expected volume, the freshness target and the valid values for key fields. A check needs a rule to check against. - Reconcile totals between systems. Compare the same figure in its source and in the warehouse on a schedule. A gap found on the day it appears is far cheaper to fix than one found at month end. - Test at each stage of the pipeline. Checks on raw data catch broken feeds. Checks on cleaned data catch failed transformations. Checks on reporting tables catch definitions that have drifted. - Track issues in one place. A shared log of what broke, why it broke and who fixed it shows you which sources cause repeat problems. Spreadsheets deserve a mention, because many issues start or end in one. In a spreadsheet you can use conditional formatting to highlight duplicates, filters to find blanks, and data validation rules to restrict what can be entered. These help with a single file. They do not help when the same data lives in five files owned by five people, which is usually the real problem. ## Why AI magnifies data quality problems A person reading a report applies judgement. They know the March figure looks low because a feed failed, or that two customer records belong to the same firm. That judgement quietly absorbs a great deal of poor data. An AI system does not have it. It uses what it is given, at speed and at volume. This changes which issues matter. A duplicate record that costs an analyst some time each week becomes a model that learns the same customer twice. A conflicting definition that people work around becomes a chatbot that gives one answer on Monday and another on Tuesday, depending on which document it retrieved. A stale knowledge base becomes an assistant that states an outdated policy with full confidence. Our guide to AI hallucination and how to reduce it explains how poor source data turns into confident wrong answers. We describe AI as a magnifying glass. Over strong foundations it amplifies the gains. Over weak foundations it speeds up failure and exposes risk. A data quality issue that was tolerable in a monthly report becomes a visible failure when an AI system acts on it many times a day. > AI does not create data quality problems. It finds the ones you already had and repeats them at scale. This is also why so many AI pilots stall. The model works in the demonstration on a clean extract, then fails on live data. Our guide to why AI projects fail sets out the pattern in more detail. ## Managing data quality as a process A one-off clean-up feels productive. The duplicates are merged, the blanks are filled and the reports agree. Within months the same issues return, because nothing that caused them has changed. Managing data quality means changing the conditions that produce bad data, not only the data itself. Four practices do most of the work. ### Give every dataset an owner Each important dataset needs a named owner in the business, not only in IT. The owner decides what the data means, what good looks like and what happens when a check fails. Without an owner, issues are logged and never resolved. ### Agree definitions once Write down what each business term means, such as active customer, revenue or on-time delivery, and get the people who use it to agree. A data dictionary records the agreement. A semantic layer enforces it, so every report and every AI system reads the same definition. ### Build quality into the architecture In a medallion architecture, data moves through three layers: bronze for raw data as it arrived, silver for cleaned and conformed data, and gold for the business-ready tables people report on. Each move between layers is a checkpoint. Tests run there, failures stop at that layer, and the gold tables contain only data that passed. This turns quality from a clean-up task into a property of the system. ### Measure and review Choose a small number of quality measures for the data that matters most, report them alongside the business figures, and review them on a fixed cycle. Our guide on how to improve data quality sets out a step-by-step approach. ## Where data quality tooling helps and stops Tools for data quality fall into a few groups. Testing frameworks let you write rules and run them in the pipeline. Observability platforms learn what normal looks like for each table and alert you when volume, freshness or distribution shifts. Master data and data quality suites handle matching, merging and cleansing at scale. Some warehouse platforms, Databricks for example, also include native constraints and monitoring. Good tooling is worth having. It runs checks you would otherwise forget, catches problems overnight and gives you a record of what failed. Our guides to automated validation with Great Expectations and rule-based testing with Soda show what this looks like in practice. Tooling does not decide what your data should mean. A tool can tell you that two revenue figures disagree. It cannot tell you which one is right, because that is a business decision. A tool can flag a stale table. It cannot make someone responsible for the feed. Buying a platform before you have owners and definitions usually produces a dashboard full of alerts that nobody acts on. Before you choose a tool, check that you can answer these questions. - Which datasets matter most, and to which decisions or systems. - Who owns each of them, by name. - What valid, complete and fresh mean for each one. - What happens, and who is told, when a check fails. If those answers exist, almost any reasonable tool will help. If they do not, no tool will. ## Where this leaves you Data quality issues are real and costly, but the ones you see are rarely the ones to fix. Duplicates, conflicting figures, stale records and broken joins all trace back to the same small set of causes: no owner, no shared definition and no checkpoint where bad data stops. Fix those at the foundation and most of the symptoms stop recurring. This matters more each year, because AI systems will use your data without the human judgement that currently hides its flaws. If you are planning AI on data you do not fully trust, the honest first step is to fix the data. Sometimes the right decision is to hold the AI work until the foundation is ready. We build the data foundation that stops these issues at source: a cloud data warehouse, a medallion architecture with checks between each layer, and a semantic layer that holds one governed definition of your business. Where the work is more about ownership, governance and analytics, our data consultancy service sets up the roles, rules and measures that keep quality from slipping. We scope honestly, and we will tell you if a targeted clean-up is all you need. Talk to us if your reports disagree and you want to know why. ## Frequently asked questions Data quality issues are flaws that stop people and systems from using data with confidence. Common examples are duplicate records, figures that disagree between systems, stale values, missing or invalid fields and joins that fail. Most are symptoms. The causes are usually missing ownership, unclear definitions and checks that run too late or not at all. Typical examples include the same customer held three times with different addresses, finance and sales reporting different revenue for the same month, a contact list full of people who have left, a report that drops rows because IDs are stored in different formats, and mandatory fields left blank. Each one traces back to a cause in ownership, definitions or validation. Profile the data behind your most used reports by checking counts, null rates, distinct values and ranges. Write down what good looks like for each important dataset, including volume, freshness and valid values. Reconcile totals between source systems and the warehouse on a schedule, test at each stage of the pipeline, and keep one shared log of issues so repeat sources become visible. Treat data quality as a process rather than a one-off clean-up. Give each important dataset a named owner in the business, agree one definition for each business term, build checks into the architecture so bad data stops between layers, and measure quality on a fixed review cycle. Tools help once owners and definitions exist, but they cannot replace them. Fix the cause as well as the symptom. Duplicates need an owner for each core entity and a match rule. Conflicting figures need one shared definition built into a semantic layer. Stale records need freshness targets and alerts. Broken joins need agreements between the teams that produce and use the data. Missing values need validation at entry and tests at load. The points most often cited are accuracy, completeness, consistency, timeliness and validity. Many frameworks add uniqueness as a sixth, which covers duplicate records. Together these dimensions give teams a shared way to describe what is wrong with a dataset and how badly. They describe symptoms, so you still need to trace each failure back to its cause. ## Frequently asked questions ### What are data quality issues? Data quality issues are flaws that stop people and systems from using data with confidence. Common examples are duplicate records, figures that disagree between systems, stale values, missing or invalid fields and joins that fail. Most are symptoms. The causes are usually missing ownership, unclear definitions and checks that run too late or not at all. ### What are some examples of poor data quality? Typical examples include the same customer held three times with different addresses, finance and sales reporting different revenue for the same month, a contact list full of people who have left, a report that drops rows because IDs are stored in different formats, and mandatory fields left blank. Each one traces back to a cause in ownership, definitions or validation. ### How to identify data quality issues? Profile the data behind your most used reports by checking counts, null rates, distinct values and ranges. Write down what good looks like for each important dataset, including volume, freshness and valid values. Reconcile totals between source systems and the warehouse on a schedule, test at each stage of the pipeline, and keep one shared log of issues so repeat sources become visible. ### How to handle data quality issues? Treat data quality as a process rather than a one-off clean-up. Give each important dataset a named owner in the business, agree one definition for each business term, build checks into the architecture so bad data stops between layers, and measure quality on a fixed review cycle. Tools help once owners and definitions exist, but they cannot replace them. ### How to fix data quality issues? Fix the cause as well as the symptom. Duplicates need an owner for each core entity and a match rule. Conflicting figures need one shared definition built into a semantic layer. Stale records need freshness targets and alerts. Broken joins need agreements between the teams that produce and use the data. Missing values need validation at entry and tests at load. ### What are the 5 points of data quality? The points most often cited are accuracy, completeness, consistency, timeliness and validity. Many frameworks add uniqueness as a sixth, which covers duplicate records. Together these dimensions give teams a shared way to describe what is wrong with a dataset and how badly. They describe symptoms, so you still need to trace each failure back to its cause. --- Shipshape Data is a London AI consultancy: we build the data foundation your AI depends on, then the AI on top. Site guide for agents: https://shipshapedata.com/llms.txt | Contact: hello@shipshapedata.com