SyncAI.news, a Varaisys broadcasting
Getting Started with DuckLake: Build a Lakehouse on DuckDB in Under an Hour
SO

Shittu Olumide

· 1 min read

EngineeringKDnuggets

Getting Started with DuckLake: Build a Lakehouse on DuckDB in Under an Hour

DuckLake is an open lakehouse table format from the DuckDB team. It keeps table metadata in a SQL database and the data in Parquet files. DuckLake 1.0 shipped in April 2026 with a stable specification and guaranteed backward compatibility. This article builds a working DuckLake lakehouse on your laptop with Python and the duckdb package: you will load Parquet data, change rows and schema, time travel, and run maintenance.

How DuckLake Works

A DuckLake has two parts. The data lives in Parquet files in a folder or object store. Everything else, meaning the table list, schemas, snapshots, and file statistics, lives in a catalog database. DuckLake can use any SQL database as its catalog, as long as it supports ACID transactions and primary key constraints. Formats like Apache Iceberg and Delta Lake track much of this in metadata files. In DuckLake, finding the right files is one SQL query.

The docs suggest a catalog by use case: DuckDB for a single client, SQLite for several local clients, and PostgreSQL for a multi-user lakehouse. We use DuckDB, the simplest option. Like other lakehouse formats, DuckLake does not support indexes, primary keys, foreign keys, or UNIQUE and CHECK constraints.

Prerequisites

DuckLake 1.0 is supported by DuckDB v1.5.2 or newer. You need a recent Python 3 and an internet connection the first time, because DuckDB downloads the ducklake extension on first use.

pip install "duckdb>=1.5.2"
python -c "import duckdb; print(duckdb.__version__)"

The first command installs DuckDB's Python package at the minimum version DuckLake 1.0 needs. The second prints the installed version, which should be 1.5.2 or higher. Create an empty project folder and put the six scripts below in it: step1_make_data.py, common.py, step2_load.py, step3_change.py, step4_time_travel.py, and step5_maintenance.py. Run them in order from that folder.

Step 1: Creating Sample Parquet Data

Step 2: Creating the Lake and Loading January

Now load January:

Original source

This story was published by KDnuggets and written by Shittu Olumide. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on kdnuggets.com

Similar News