Welcome to pg_mooncake on Mooncake

Architecture

Mon, 01 Jan 0001 00:00:00 +0000

pg_mooncake (v0.3.0) combines three specialized components inside a single PostgreSQL deployment:

pg_duckdb (v1.2.0): Embeds the DuckDB v2.0.0 vectorized analytical execution engine inside PostgreSQL and routes queries against non-heap table access methods.
moonlink (v0.0.1): A Rust library and background worker embedded inside pg_mooncake that ingests PostgreSQL logical replication streams, maintains an in-memory Apache Arrow write buffer and Roaring Bitmap deletion vectors, flushes Parquet files, and emits Apache Iceberg v2 table metadata.
duckdb_mooncake (v0.0.2): A DuckDB 2.0 C++/Rust storage extension loaded automatically by pg_mooncake from a local extension repository (mooncake.extension_repository). It connects to Moonlink over a local Unix domain socket RPC (pg_mooncake/moonlink.sock) to read consistent snapshots across both flushed Parquet files and buffered Arrow batches.

Together, they expose a native PostgreSQL Table Access Method (USING mooncake) managed through CALL mooncake.create_table(dst, src).

Moonlink Ingestion Engine

Mon, 01 Jan 0001 00:00:00 +0000

Moonlink (pgsty/moonlink, current release v0.0.1) is a Rust library and standalone service designed to write and maintain real-time Apache Iceberg v2 and Delta Lake tables for operational systems and event streams without Apache Spark or Kafka.

Inside pg_mooncake, Moonlink runs as an embedded PostgreSQL background worker (mooncake_bgworker) communicating over a local Unix domain socket ($PGDATA/pg_mooncake/moonlink.sock). It can also be compiled and deployed as a standalone HTTP REST and gRPC ingestion server (moonlink_service).

duckdb_mooncake & DuckDB 2.0 Integration

Mon, 01 Jan 0001 00:00:00 +0000

duckdb_mooncake (pgsty/duckdb_mooncake, current release v0.0.2, MIT license) is a C++ and Rust FFI storage extension for DuckDB v2.0.0. It allows DuckDB (both standalone and embedded inside PostgreSQL via pg_duckdb 1.2.0) to attach directly to a moonlink metadata directory and execute vectorized scans over Moonlink tables.

Role Inside `pg_mooncake 0.3.0`

When pg_mooncake executes a query against a USING mooncake table inside PostgreSQL, pg_duckdb delegates the table scan to DuckDB. pg_mooncake automatically initializes the DuckDB session by running:

v0.1 vs v0.2 & v0.3 Evolution

Mon, 01 Jan 0001 00:00:00 +0000

pg_mooncake has evolved through three major architectural milestones:

v0.1 (Legacy Delta Lake Engine): Introduced direct CREATE TABLE ... USING columnstore tables backed by Delta Lake and DuckDB (pg_duckdb).
v0.2 (Moonlink & Apache Iceberg Redesign): Replaced direct columnstore writes with a columnstore mirror architecture (CALL mooncake.create_table(dst, src) + USING mooncake) powered by moonlink logical replication, in-memory Arrow write buffers, Roaring deletion vectors, and Apache Iceberg v2 metadata.
v0.3.0 (PGSTY Maintained Release): Upgraded pg_mooncake to pgrx 0.19.3 (PostgreSQL 14–18), ported duckdb_mooncake v0.0.2 to pg_duckdb 1.2.0 and DuckDB v2.0.0, fixed Moonlink (v0.0.1) snapshot LSN hangs and partitioned table replica identity handling, and patched 20+ RustSec dependency advisories.

Looking for v0.1 Documentation?

Complete documentation for the legacy v0.1 release (USING columnstore, mooncake.create_secret, mooncake.columnstore_tables, mooncake.read_parquet) is preserved in the v0.1 Archive (Delta Lake).

Welcome to pg_mooncake on Mooncake

Architecture

Moonlink Ingestion Engine

duckdb_mooncake & DuckDB 2.0 Integration

Role Inside pg_mooncake 0.3.0

v0.1 vs v0.2 & v0.3 Evolution

Role Inside `pg_mooncake 0.3.0`