Guest Article: This article was originally published on HackerNoon by Shawn Gordon and is republished here with permission from the author.
I previously explored "What the Heck is dbc?" which led to a conversation with Philip Moore, a Voltron Data alumnus. Moore founded GizmoData, working on projects including GizmoEdge and GizmoSQL. This article addresses what GizmoSQL does, why it's interesting, and its practical applications.
Background
GizmoSQL is an open-source SQL database engine powered by DuckDB and Apache Arrow Flight SQL. DuckDB is "an open-source, in-process analytical database engine designed for OLAP workloads" that executes queries directly within applications without requiring separate servers.
Apache Arrow Flight SQL is "a protocol layered on Arrow Flight RPC that enables clients to execute standard SQL queries against remote database servers," with results streamed in the efficient Arrow columnar format. The Apache Arrow ecosystem provides a cross-language in-memory analytics platform with standardized columnar memory format, enabling zero-copy reads and efficient data sharing across systems.
What is GizmoSQL?
Fundamentally, GizmoSQL is a small server running DuckDB with Arrow Flight SQL protocol wrapped around it for remote DuckDB execution. While DuckDB handles billions of rows on a laptop, running it in cloud VMs with substantial computational resources enables processing trillions of rows.
According to GizmoData, they completed the Trillion Row Challenge in 2 minutes for just 9 cents!
Digging In
A free demo offers TPC-H dataset with prewritten queries. The interface displays query history with execution times, and clicking historical queries reloads them without copy-pasting.
Testing the 1.5 million-row customer table with a query counting records where c_nationkey equals 15 returned 60,000 results in 154 milliseconds.
Query 22 (Global Sales Opportunity) executed under one second with substantial processing.
Query 2 completed in approximately half a second.
Summary
GizmoData combined existing technologies with innovation to create a straightforward product delivering exceptional speed and usability. The service supports all major cloud platforms, including OCI. While similar to MotherDuck, it differentiates through Arrow Flight SQL integration. The platform offers integrations and adapters for expanded functionality.
I'm enthusiastic about this solution's potential workplace impact, particularly for accelerating previously time-intensive database operations.
This article was originally published on HackerNoon by Shawn Gordon and is republished here with permission.