iceberg on Mooncake

pg_mooncake 0.3.0: First PGSTY Release with DuckDB 2.0, pgrx 0.19.3, and PostgreSQL 14–18 Support

Thu, 05 Nov 2026 00:00:00 +0000

pg_mooncake 0.3.0 — released alongside duckdb_mooncake v0.0.2 and moonlink v0.0.1 — is the first release of the Mooncake real-time analytics suite maintained by PGSTY:

pgsty/pg_mooncake v0.3.0 (MIT)
pgsty/duckdb_mooncake v0.0.2 (MIT)
pgsty/moonlink v0.0.1 (BSL-1.1 with PG_MOONCAKE LINKING & DISTRIBUTION EXCEPTION, client crates MIT)

Binary RPM and DEB packages for PostgreSQL 14, 15, 16, 17, and 18 across RHEL / Rocky / AlmaLinux 8, 9, 10 and Debian 12/13, Ubuntu 22.04/24.04 (x86_64 and aarch64) are available through the PGEXT repository and Pigsty.

Mooncake: a real-time storage engine for Apache Iceberg

Mon, 11 Aug 2025 00:00:00 +0000

This technical overview assumes familiarity with Apache Iceberg, an open table format for analytical datasets. Iceberg is part of the general “Lakehouse” movement, where users:

Store large datasets using open file formats (Parquet + Metadata) in Object Storage
Query them directly by engines like Spark, Trino, Flink, and DuckDB using SQL
Get the benefits of ‘database features’ like schema evolution & time-travel
Scale practically infinitely, and aren’t locked into a vendor

Iceberg was built for slow-changing data + batch analytics & processing. That world is pretty clear and only improving with engines getting better at reading open formats.

No, Iceberg won’t fix your data stack.

Mon, 21 Jul 2025 00:00:00 +0000

Life is expectations vs reality. Here is one for Apache Iceberg – a favorite amongst the data community.

Expectations: Iceberg is the solution to all my data pains

See, the core of Iceberg is pretty neat:

Keep all data in object storage in open file formats like Parquet.
Fool any execution engine (DuckDB, ClickHouse, Spark, Snowflake, etc.) into thinking these files are a table and query them with SQL directly.
[Ongoing progress] Get exceptional performance on these tables, without having to load data into a system-specific format.

And with clever design you get:

but, my Postgres (+ lakehouse) is free

Fri, 13 Jun 2025 00:00:00 +0000

Disclaimer: Your Postgres and Lakehouse are free and open too. You don’t need to be (or spend like) a giant to use them.

Why Postgres + Lakehouse?

Reason #1: If you believe in AI, this is the agent stack

Agents need memory. Both working memory and longer-term knowledge.

Working memory has to be instant, reliable, and accurate, just like app state. What’s the best place to store it? Postgres. That’s true for both apps and agents. This state is typically tied to an individual user, tenant, or agent.

pg_mooncake v0.2: what we're cooking

Tue, 11 Feb 2025 00:00:00 +0000

With v0.1, we proved that Postgres can deliver great analytics performance. Our approach combined two key pieces—DuckDB vectorized engine and a columnstore table—making pg_mooncake the only extension in Postgres that integrates both.

The results speak for themselves: pg_mooncake ranks in ClickBench’s top 10, and we built the fastest Parquet-based engine along the way.

Now, with v0.2, we’re tackling the next big challenge; we call it out-of-the-box analytics.

First, why in-Postgres analytics?

Applications often have their first analytics needs far before they have their first data warehouse or analytics system.

wth are S3 tables

Sun, 12 Jan 2025 00:00:00 +0000

Amazon announced table buckets during re:Invent last year. An S3 bucket that’s also a table? What does that mean?

TL;DR: S3 tables allow Iceberg tables to exist without a catalog. Let me explain.

Iceberg vs Delta Lake

What? I thought Iceberg and Delta are just table formats? Why are we comparing the way they’re implemented?

A key difference between Iceberg and Delta Lake is “Delta tables can exist without a catalog, but Iceberg cannot”, due to their metadata design:

pg_mooncake: Fast Analytics in Postgres with Columnstore Tables and DuckDB

Wed, 06 Nov 2024 00:00:00 +0000

pg_mooncake brings columnstore tables with DuckDB execution to Postgres for fast analytics.

It supports transactional inserts, updates, and deletes, just like regular Postgres tables. Data is written with Delta Lake (and Apache Iceberg in v0.2+) table metadata, allowing it to be queried outside of Postgres.

Why another Postgres analytics extension?

Postgres is the most popular database these days, but it’s bad at analytics. Numerous attempts have been made to address this, but with limited success so far. Fast analytics require a distinct set of techniques, such as columnar storage, vectorized execution, and late materialization, etc. Early projects like Citus & Timescale added columnar storage, but there was no good vectorized execution to leverage the format.

Reflections from the Iceberg Meetup

Tue, 08 Oct 2024 00:00:00 +0000

Last month, I spoke at the Iceberg meetup at Snowflake HQ. Public speaking has never been my favorite, and knowing Snowflake CEO Sridhar and Tabular CEO Ryan were in the audience did not help.

It was day 18 of starting Mooncake Labs, and I spoke about a general indexing framework for Iceberg. You can find the details here. Here were some of my learnings:

1. Great products are a reflection of its community.

I spent the last decade building a proprietary database. Every time a feature request came in, engineering and product leaders would spend hours prioritizing its impact. Almost every time, it boiled down to one question: How much ARR? I watched my best engineers spend months building awkward hooks and extensions that did not fit the bigger picture. I often thought, Just ask the customer to make a PR :) The end product of this type of prioritization process tends to look like:

Things we learnt

Sun, 08 Sep 2024 00:00:00 +0000

The title should really be “Things we learnt, the hard way”.

As we start this new journey, we spent some time thinking about the axioms of data today. These are things we (or no one for that matter) can dispute.

1. If Postgres can do it, Postgres will do it

We’ve seen this one plenty of times now – it started with JSON, then moved to geo-spatial (PostGIS), and now we’re seeing it with vector search (pgvector).