GizmoSQL + DuckLake: Elastic Compute Over Shared State

Scaling stateless compute is easy. Scaling independent compute nodes that all need to share the same mutable data state is more interesting, and it is the reality of modern analytics.

GizmoData runs analytical SQL workloads on independent GizmoSQL nodes. A cluster might run one node at low concurrency, grow to five as demand climbs, and shrink back down when that extra capacity is no longer needed.

This is one of the scenarios where GizmoSQL and DuckLake really shine together.

Shared Catalog, Shared Data

DuckLake keeps table data in ordinary Parquet files and stores the lakehouse metadata in a transactional SQL catalog. That catalog tracks schemas, tables, columns, data files, snapshots, statistics, and everything else needed to interpret the files underneath it. For a multi-user lakehouse with remote clients, DuckLake’s own guidance is to use PostgreSQL as that catalog database.

In a GizmoData deployment, multiple GizmoSQL nodes connect to the same DuckLake catalog and the same object storage:

A new node connects directly to the existing catalog and object storage. It does not copy the lakehouse or rebuild its metadata before it can serve queries. That matters when nodes are ephemeral: GizmoData can add compute to absorb a surge, then remove it when demand falls, without moving or duplicating the underlying data. Scaling stays fast and operationally simple, and you avoid the storage cost, synchronization work, and consistency risk that come with maintaining separate copies of the same data.

Concretely, every node runs the same handful of statements to join the lakehouse. This is the setup from the GizmoSQL DuckLake integration guide, and it is identical whether the node is the first one or the fifth:

INSTALL ducklake; LOAD ducklake;
INSTALL postgres; LOAD postgres;

-- The shared catalog
CREATE OR REPLACE SECRET postgres_secret (
  TYPE postgres,
  HOST 'localhost',
  PORT 5432,
  DATABASE 'ducklake_catalog',
  USER 'postgres',
  PASSWORD 'your_password'
);

-- The shared data
CREATE OR REPLACE SECRET ducklake_secret (
  TYPE DUCKLAKE,
  METADATA_PATH '',
  DATA_PATH 's3://my-bucket/lakehouse/',
  METADATA_PARAMETERS MAP {
    'TYPE': 'postgres',
    'SECRET': 'postgres_secret'
  }
);

ATTACH 'ducklake:ducklake_secret' AS my_lakehouse;
USE my_lakehouse;

Compute scales with demand while the shared data state stays put.

The Nodes Can Write Too

This is not just shared read access.

Multiple GizmoSQL nodes can read from and write to the same DuckLake-backed tables. DuckLake provides ACID transactions with snapshot isolation, and it handles concurrent writers the way you would expect from a transactional system: when two transactions commit changes that do not logically conflict (concurrent inserts into the same table, for example), the later one is retried against the catalog without rewriting any data files. When the changes genuinely conflict, such as two transactions deleting rows from the same data file, one of them is aborted and the client is told why. The details are in DuckLake’s conflict resolution documentation.

That gives GizmoData an important property: adding a node adds another independent compute process that can immediately participate in work against the same transactional lakehouse.

There is no need to create a separate data copy for each node, and nothing to reconcile afterward. Every node reads and writes the same set of snapshots.

Why the Catalog Matters

The catalog is more than a list of schemas.

It holds enough structural information for every node to operate against the same representation of the lakehouse: which tables exist, which files belong to them, what the current schema is, and which snapshot is visible right now. Because that state lives in one transactional database rather than in each node’s memory or local disk, a node carries no state worth preserving. It can be created or destroyed at will.

That makes autoscaling cleaner. GizmoData scales the compute tier independently, while DuckLake maintains the persistent, shared state underneath it.

DuckLake provides the shared transactional lakehouse.

GizmoSQL provides the elastic SQL compute on top.