v0.1 vs v0.2 & v0.3 Evolution

pg_mooncake has evolved through three major architectural milestones:

1. v0.1 (Legacy Delta Lake Engine): Introduced direct CREATE TABLE ... USING columnstore tables backed by Delta Lake and DuckDB (pg_duckdb).
2. v0.2 (Moonlink & Apache Iceberg Redesign): Replaced direct columnstore writes with a columnstore mirror architecture (CALL mooncake.create_table(dst, src) + USING mooncake) powered by moonlink logical replication, in-memory Arrow write buffers, Roaring deletion vectors, and Apache Iceberg v2 metadata.
3. v0.3.0 (PGSTY Maintained Release): Upgraded pg_mooncake to pgrx 0.19.3 (PostgreSQL 14–18), ported duckdb_mooncake v0.0.2 to pg_duckdb 1.2.0 and DuckDB v2.0.0, fixed Moonlink (v0.0.1) snapshot LSN hangs and partitioned table replica identity handling, and patched 20+ RustSec dependency advisories.

Architectural Comparison

Why v0.2 / v0.3 Moved Away from v0.1’s Write Path

Limitations of v0.1

In v0.1, writing directly to a USING columnstore table worked well for large batch loads (100,000–1,000,000 rows per transaction), but struggled in real-world operational PostgreSQL workloads:

• No indexes or constraints: You could not create primary keys, unique constraints, or B-tree indexes on a USING columnstore table.
• Small-file amplification: Every INSERT or UPDATE transaction created a new Parquet file, quickly degrading scan performance under OLTP traffic.
• DuckDB write-path coupling: Routing transactional writes through DuckDB inside PostgreSQL’s executor created memory and transaction-handling bottlenecks.

How v0.2 and v0.3.0 Solve This

By decoupling the OLTP write path (standard PostgreSQL heap tables) from the analytical storage and compute path (moonlink + pg_duckdb 1.2.0 + duckdb_mooncake v0.0.2):

1. Zero Impact on OLTP: Your primary tables keep all PostgreSQL features—primary keys, foreign keys, B-tree/GIN/vchord indexes, triggers, and row-level locking.
2. High-Throughput Mutations: moonlink absorbs high-frequency INSERT, UPDATE, and DELETE streams in memory using Arrow buffers and Roaring deletion vectors, flushing optimized Parquet files in the background.
3. Schema Evolution: ALTER TABLE ... ADD COLUMN and DROP COLUMN on the source heap table automatically propagate to the columnstore mirror.
4. Remote Replication (src_uri): You can even run pg_mooncake on a dedicated analytical PostgreSQL instance and mirror tables from a remote production PostgreSQL database (including RDS, Aurora, Supabase, or Neon) via src_uri => 'postgresql://...'.