Ask five people at your company what a data lakehouse actually is and you will get five different half-right answers. That confusion is expensive: teams keep building a lake and a warehouse side by side, paying twice for infrastructure that was supposed to be one system.
We build data foundations at Shipshape Data, and the lakehouse question comes up in nearly every engagement now. Someone has read that lakehouses fix the lake-versus-warehouse problem, wants one, and has no clear picture of what they are actually asking for. This guide sets that out properly: what a lakehouse is, why it matters, how the five layers fit together, how to plan one, and where projects tend to go wrong. Read it end to end if you are starting from scratch, or jump to the layer breakdown if you already know the shape and want the detail.
What a data lakehouse actually is
A data lakehouse combines the flexibility and low cost of a data lake with the reliability and query performance of a data warehouse. You store everything, structured tables, semi-structured logs, unstructured documents, in open file formats on cheap cloud object storage, and you still get fast SQL, proper transactions, and enforced data quality on top of it. That is the whole idea in one sentence, and it is worth sitting with, because most of the confusion around lakehouses comes from people missing one half of it.
A plain data lake gives you the cheap, flexible storage but none of the guarantees. Anyone can write anything to it, schemas drift, two jobs writing at once can corrupt a table, and six months in nobody trusts what is actually there. A data warehouse gives you the guarantees but charges warehouse prices for storage and locks your data into a format only that warehouse understands, which makes running a machine learning model against it needlessly painful. A lakehouse is the attempt to get both without the drawbacks of either, and modern table formats like Delta Lake, Apache Iceberg and Apache Hudi are what make it possible: they sit on top of plain files in object storage and add the transaction log, schema enforcement and versioning that used to require a warehouse.
The practical result is that you stop copying data between two platforms to serve two audiences. Your analysts query the same tables your data scientists train models against, both read from one governed source, and the constant extract-transform-load jobs that used to shuffle data from lake to warehouse mostly disappear.
Why it matters beyond the infrastructure diagram
Running a lake and a warehouse side by side is not just architecturally messy, it costs real money and real time. You store the same data twice. You run pipelines to keep the copies in sync, and those pipelines break, and someone has to fix them. Analysts query one version while data scientists query another, slightly different, version, and eventually somebody notices the numbers do not match and spends a week finding out why.
A unified lakehouse removes most of that overhead directly. One copy of the data, one set of access controls, one place to enforce quality rules. Organisations that consolidate typically see deployment cycles shrink by weeks, not because the lakehouse itself is faster to build against but because engineers stop spending their time on data plumbing between two systems and start spending it on the problem they were actually hired to solve.
The governance angle people underrate
The part that gets less attention than cost is governance. A traditional data lake struggles to give you a proper audit trail, because data arrives from everywhere in whatever format it turns up in, and nobody enforced a schema on the way in. A lakehouse's metadata layer tracks every write, every schema change and every table version, which means when your compliance team asks where a number in a report actually came from, you can answer with a query rather than a guess. We have sat in enough audit prep meetings to know how much that single capability is worth when the pressure is on.
A unified architecture lets a business respond to change faster, because the whole team is working from the same source of truth instead of reconciling two versions of the same numbers.
How to plan a lakehouse before you touch any tooling
The projects that go well start with an honest look at the current estate, not a vendor comparison spreadsheet. Skip that step and you end up with an architecture that looks impressive in a slide deck and does not fix any of the problems your teams actually have.
Map what you already have
Write down every source that feeds your analytics, reporting and machine learning today: the operational databases, the SaaS tools with an API, the streaming platforms, the file shares nobody officially owns. Work out your total data volume and which datasets are growing fastest, because that shapes your storage and compute costs more than almost anything else. Then talk to the people who actually use the data. Ask which queries take forever, which reports genuinely matter, and where they waste time reconciling numbers that should already agree. You will usually find data copied between three or four systems, each copy a little stale, each one somebody's job to keep updated by hand.
Write outcomes down, not aspirations
"Modernise our data platform" is not a target, it is a mood. Write something you can actually measure: cut the time from ingestion to analyst access from two days to two hours, or support fraud detection on live transaction streams instead of yesterday's batch. Concrete targets like these do two jobs at once. They shape the architectural decisions that follow, and they give you something to point at when a stakeholder asks what the investment actually bought.
Pick storage and a table format that match your team
Most lakehouses sit on Amazon S3, Azure Data Lake Storage or Google Cloud Storage, chosen mainly by where your data already lives and which cloud your engineers already know. On top of that you pick a table format, most commonly Delta Lake, Apache Iceberg or Apache Hudi, and this is the layer that actually turns a lake into a lakehouse: it adds the transaction log, the schema enforcement and the versioning that a folder full of Parquet files does not have on its own. The three formats overlap a great deal in what they promise. The real decision usually comes down to which processing engine your team already runs and how much of that ecosystem's tooling you want to inherit for free.
The five layers that make a lakehouse work
Strip away the vendor branding and every lakehouse is built from the same five layers, each doing one job and handing off to the next. Understanding them properly is what lets you design something that fits your actual workload instead of copying whatever a case study did.
Ingestion layer
This is where data enters the system, pulled from operational databases, APIs, streaming platforms and file stores and loaded into lakehouse storage. It has to handle both batch and real-time flows at once, a nightly export from your billing database sitting alongside a continuous stream of clickstream events from your website. Teams typically lean on something like AWS Database Migration Service for relational sources, Kafka for streaming, and hand-built connectors for the SaaS tools that only expose a REST API.
The part people skip, and regret skipping, is validating quality on the way in. Schema checks, type validation, basic business-rule enforcement, all of it belongs here, catching a malformed record before it lands rather than after an analyst has already built a dashboard on top of it.
Storage layer
Raw and processed data both live here, on low-cost object storage, written in an open format such as Parquet or ORC that any tool can read without a proprietary licence. The thing that actually saves money is separating storage from compute: you pay for processing power only when a query runs, not for servers sitting idle waiting for the next report. You organise data into partitions that match how it actually gets queried, which is what keeps performance sane as the volume grows into the terabytes.
Metadata layer
This is the layer that turns a lake into a lakehouse, and it is worth understanding properly rather than treating as a black box. It tracks which files belong to which version of which table, enforces ACID transactions so two jobs writing at the same time cannot corrupt each other, and holds the schema definitions that keep everyone's data consistent. Delta Lake, Apache Iceberg and Apache Hudi all do this job, each with its own take on the transaction log, while keeping your actual data in open formats underneath.
It also adds features a plain lake cannot offer: time travel, so you can query a table exactly as it looked last Tuesday, and zero-copy cloning, so you can spin up a full development copy of a dataset in seconds instead of physically duplicating terabytes. Quality rules, lineage tracking for audits and access control all live at this layer too, rather than being bolted on through separate tools nobody quite trusts.
API layer
The API layer is what lets analytics tools and applications query your data without needing to understand how the storage actually works underneath. It translates SQL, DataFrame calls and REST requests into optimised reads against object storage, and does the caching and query planning that closes the performance gap between reading from cloud storage and reading from a warehouse's own disks. Business analysts hit this through familiar SQL tools like Tableau or Power BI; data scientists hit the same data through Python or Spark. Same tables, different doors in.
Consumption layer
This is where people actually use the thing: BI dashboards, notebook environments, machine learning frameworks, all reading directly from lakehouse storage through the API and metadata layers below. Nobody is working from a third copy that went stale on Tuesday. You set access permissions and resource quotas here so one analyst running a heavy query at 9am does not slow everyone else down, and result caching keeps repeated queries fast without extra engineering.
Common patterns: medallion and lambda
Two patterns turn up in almost every lakehouse you will see, and it helps to know both before you draw your own diagram.
Medallion architecture
Medallion organises data into bronze, silver and gold layers, each one more refined than the last. Bronze holds raw data exactly as it arrived, unedited. Silver holds the same data cleaned, validated and joined into something usable. Gold holds aggregated tables built for a specific business use case, the numbers that actually land in a dashboard. The value of this pattern is debugging: when a number looks wrong, you can trace it back through each stage instead of guessing where a transformation went sideways.
Lambda architecture
Lambda runs a batch pipeline and a streaming pipeline in parallel rather than picking one. The streaming path gets you a fast, approximate answer in near real time. The batch path runs overnight, reconciles anything that arrived late, and gives you the accurate final answer by morning. You get speed and correctness, just not from the same pipeline at the same moment, which is a trade-off worth being explicit about with whoever is waiting on the numbers.
Most organisations end up leaning on medallion for how they structure data quality and lambda for how they structure timing, and the two combine without much friction: bronze and silver can run on the batch path, gold can be served fast enough for near-real-time dashboards once the aggregation logic is defined.
Lakehouse versus lake versus warehouse
The three architectures trade off cost, governance and performance differently, and picking between them without understanding the trade-off is how projects end up disappointed six months in.
- A data warehouse gives you fast SQL and strong consistency on structured data, but you pay premium prices for storage you cannot separate from compute, and the proprietary format makes running machine learning against the same data expensive and awkward.
- A data lake solves the cost problem: any format, cheap object storage, no premium. What it sacrifices is governance. Nothing enforces a schema or stops two jobs writing at once, so teams end up spending real time just validating that the data is trustworthy before they use it.
- A lakehouse keeps the lake's storage economics and open formats, then adds the warehouse's ACID transactions and schema enforcement back on top through the metadata layer. Analysts and data scientists work from the same tables, so the ETL overhead of keeping two systems in sync mostly goes away.
None of the three is universally right. A small team running mostly structured reporting with no machine learning ambitions may genuinely be better off with a straightforward warehouse, and building a lakehouse for that use case is over-engineering. The lakehouse earns its complexity when you actually have mixed workloads, structured and unstructured data, BI and ML, batch and streaming, all wanting access to the same underlying facts.
Where lakehouse projects actually go wrong
We see the same handful of mistakes on repeat. The most common is treating the lakehouse as a technology purchase rather than an architecture decision: a team picks Databricks or Snowflake, stands up the platform, and only then works out what data is going into it and who owns what. That order produces a very expensive, very well-engineered mess.
The second is skipping governance until something forces the issue, usually an audit or a security incident. Access control and lineage are far cheaper to build in from day one than to retrofit onto a system that thirty pipelines already depend on. The third is underestimating the migration itself. Moving from an existing warehouse to a lakehouse is not a weekend job; it is months of careful, table-by-table work, and teams that budget for a quick lift-and-shift are usually the ones still mid-migration a year later.
The fourth, and the one we spend the most time fixing on client engagements, is building the lakehouse before anyone has mapped the existing estate properly. You cannot design good partitioning, sensible access boundaries or a sane ingestion strategy for data you have not actually looked at yet. Skip that step and the coverage gaps only turn up during an outage or an audit, which are the two worst possible moments to discover them.
Getting started
If you take one thing from this guide, make it this: a lakehouse is worth building when your teams genuinely need mixed workloads on the same governed data, and it is a waste of budget when a straightforward warehouse already does the job. Map your current estate honestly, write down outcomes you can actually measure, then choose your storage and table format to match the team you have, not the team you wish you had.
If you want a clear picture of where your own gaps are before you spend anything on tooling, that is exactly the kind of assessment we run. Talk to us and we will tell you straight whether a lakehouse is the right next step for your data, or whether the real problem sits somewhere else entirely.