v0.4.0-alpha.1Dex-only identity and one-command GitHub App setup.Read the changelog →

OpenLakeForge

Open-source lakehouse · Apache-2.0 · Alpha

Your modern lakehouse,
forged in one command.

OpenLakeForge is the open-source lakehouse for small data teams. Ingestion, validation, Iceberg storage, SQL, orchestration, governance and dashboards come assembled, wired and deployed, on your laptop, AWS or Azure.

$ uv tool install openlakeforge$ olf init   # openlakeforge.yaml + lakehouse_code/$ olf deploy --provider local   # kind cluster + the whole platform
Quick start Star on GitHub

Assembled from open source you already know

How it works

One platform, from raw files to dashboards.

Data moves through three medallion layers, each owned by one tool. Dagster orchestrates every run; around that path sit the catalog, the query engine and, when you want them, governance and dashboards.

OpenLakeForge architecture: Dagster orchestrates dlt, Floe and dbt-trino over Bronze, Silver and Gold Iceberg tables in object storage, queried by Trino and Superset and governed by OpenMetadataOpenLakeForge architecture: Dagster orchestrates dlt, Floe and dbt-trino over Bronze, Silver and Gold Iceberg tables in object storage, queried by Trino and Superset and governed by OpenMetadata
Bronze

Source files land as CSV in object storage.

dlt
Silver

Technical contracts validate them into Iceberg tables.

Floe
Gold

Business marts are modelled in SQL and run in the query engine.

dbt-trino on Trino

Why OpenLakeForge

Less platform work, same modern lakehouse.

Small teams

Built for small data teams

Less operational work to run a modern lakehouse.

Modular

Start with the core

Run the core data path alone, add governance and dashboards when you need them.

Self-hosted

Open and yours

Your infrastructure, your storage, your data. Apache-2.0.

Portable

Deploy anywhere

Local Kubernetes, AWS and Azure share the same platform contracts.

Open format

Apache Iceberg

Storage stays independent from the engines that read it.

Batteries included

Infrastructure included

Kubernetes, Terraform and Helm ship with the platform, pinned and tested.

Everything as code · AI-ready

Every layer is code.
Every layer is AI-ready.

Infrastructure, data contracts, SQL, pipelines, dashboards and CI are all declarative files in Git. An AI agent reads the same source of truth as your team, proposes changes as pull requests, and the platform validates them like any other change.

Each platform layer and the files that describe it
InfrastructureTerraform + Helm, pinned in the releaseinfra/ · component-catalog.yaml
PlatformProvider, stages, preset, identityopenlakeforge.yaml
Data productsDomains, products, sourceslakehouse.yaml · source.yaml
Bronze → SilverIngestion and technical contractsbronze/ · silver/
GoldBusiness marts in SQLgold/ (dbt)
OrchestrationOne job per productpipelines/dagster/
DashboardsReports over Golddashboards/superset/
DeliveryBuild once, promote DEV → PRODGitHub Actions · revisions
Agent sessionIllustrative
> Add our daily orders export and a revenue-by-region mart.reads   openlakeforge.yaml, lakehouse.yaml$ olf source new orderswrites  silver/sales/orders.yaml   # Floe contract$ olf product new sales/revenue_by_regionwrites  gold/revenue_by_region/models/*.sqlopens   pull request #42✓ build-revision   sha256:3f9c…✓ deploy-stage dev · e2e passed… prod waits for your approval

Stages

From a commit to PROD, built once.

A change is built once into an immutable project revision, activated in DEV, tested, approved, then activated unchanged in PROD. PROD never rebuilds, and no data is copied between stages.

Stage promotion: a merged pull request builds one image and one project revision, which is activated in the DEV namespace, tested, approved, and then activated unchanged in PRODStage promotion: a merged pull request builds one image and one project revision, which is activated in the DEV namespace, tested, approved, and then activated unchanged in PROD

Your project

You own the data products. The platform stays in the box.

olf init creates a writable project. Terraform, Helm and images stay in the installed distribution, so your repository holds only what is yours.

Slim Default

Ingestion, validation, Iceberg, catalog, Trino, dbt and Dagster.

The shortest path to a working lakehouse.

Full

Slim + OpenMetadata governance + Superset dashboards.

You also want lineage, a catalog UI and dashboards.

my-lakehouse/├── openlakeforge.yaml         # provider, stages, preset└── lakehouse_code/    ├── lakehouse.yaml         # domains and data products    ├── bronze/                # sources and their dlt loaders    ├── silver/                # Floe contracts per domain    ├── gold/                  # dbt projects per product    ├── pipelines/dagster/     # one job per product    └── dashboards/superset/

Deploy anywhere

Same contracts on your laptop and in the cloud.

Local

kind on Docker

The whole platform on your machine. Plain HTTP on *.olf.localhost, no certificates.

olf deploy --provider local

AWS

EKS, S3, RDS, Glue

Managed Kubernetes, storage, database and catalog behind the same contracts.

olf deploy --provider aws

Azure

AKS and ACR

The in-cluster platform services on Azure Kubernetes Service.

olf deploy --provider azure

Is it a good fit?

Honest about where it stands.

A good fit

  • A small team that wants a working batch lakehouse without building the platform first
  • Keeping data on your own Kubernetes, storage and cloud account
  • Moving the same project between a laptop, AWS and Azure

Not yet

  • Streaming workloads: v1 is batch-first
  • Production-critical use: the project is alpha
  • Teams that want a fully managed SaaS

Get started

The lake is yours.
We bring the forge.

Docker and uv are all you need. olf provisions its own pinned Terraform, Helm, kubectl and kind.

Read the quick start Browse the docs