TypeScript + DuckDB
Generate asynchronous TypeScript code for DuckDB databases using the @duckdb/node-api driver (DuckDB Node.js Neo). Includes support for DuckDB’s advanced types like structs, lists, and maps.
Overview
Section titled “Overview”| Property | Value |
|---|---|
| Generator | typescript/duckdb (or typescript/duckdb/node-api) |
| Runtime | Node.js |
| API Style | Asynchronous (async/await) |
| Driver | @duckdb/node-api |
Installation
Section titled “Installation”# Install SQG (choose one)pnpm add -D @sqg/sqg # pnpmnpm install -D @sqg/sqg # npmyarn add -D @sqg/sqg # yarn
# Install runtime dependencypnpm add @duckdb/node-api # or: npm install / yarn addExample Configuration
Section titled “Example Configuration”version: 1name: my-app
sql: - files: - queries.sql gen: - generator: typescript/duckdb output: ./src/generated/Quick Start
Section titled “Quick Start”-
Initialize a project
Terminal window sqg init --generator typescript/duckdb -
Write your SQL
-- MIGRATE 1CREATE TABLE users (id INTEGER PRIMARY KEY,name VARCHAR NOT NULL,email VARCHAR UNIQUE,metadata STRUCT(role VARCHAR, active BOOLEAN),tags VARCHAR[],created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);-- TESTDATA seedINSERT INTO users (id, name, email, metadata, tags){'role': 'admin', 'active': true},['developer', 'lead']);-- QUERY all_usersSELECT * FROM users ORDER BY created_at DESC;-- QUERY get_user :one@set id = 1SELECT * FROM users WHERE id = ${id};-- QUERY get_user_tags :one :pluck@set id = 1SELECT tags FROM users WHERE id = ${id};-- EXEC create_user@set id = 2@set name = 'Bob'INSERT INTO users (id, name, email) VALUES (${id}, ${name}, ${email}); -
Generate code
Terminal window sqg sqg.yaml -
Use the generated code
import { DuckDBInstance } from '@duckdb/node-api';import { MyApp } from './generated/my-app';async function main() {// Create database connectionconst instance = await DuckDBInstance.create(':memory:');const connection = await instance.connect();// Run migrationsfor (const migration of MyApp.getMigrations()) {await connection.run(migration);}// Create query instanceconst queries = new MyApp(connection);// Insert data// Query dataconst users = await queries.allUsers();console.log(users);const user = await queries.getUser(1); // Gets Alice from TESTDATAconsole.log(user?.name); // 'Alice'// Access complex typesconsole.log(user?.metadata?.entries.role); // 'admin'console.log(user?.tags?.items); // ['developer', 'lead']}main(); -
See the generated code
// ...async allUsers(): Promise<{id: number | null;name: string | null;email: string | null;metadata: { entries: { role: string | null; active: boolean | null } };tags: { items: (string | null)[] };created_at: { micros: bigint } | null;}[]> {const sql = "SELECT * FROM users ORDER BY created_at DESC;";const reader = await this.conn.runAndReadAll(sql, []);return reader.getRowObjects() as {id: number | null;name: string | null;email: string | null;metadata: { entries: { role: string | null; active: boolean | null } };tags: { items: (string | null)[] };created_at: { micros: bigint } | null;}[];}// ...
Complex Types
Section titled “Complex Types”DuckDB supports advanced types that SQG fully handles:
Structs
Section titled “Structs”SQL:
CREATE TABLE products ( id INTEGER, details STRUCT(name VARCHAR, price DECIMAL(10,2), in_stock BOOLEAN));TypeScript:
interface DetailsStruct { entries: { name: string | null; price: { width: number; scale: number; value: bigint } | null; in_stock: boolean | null; };}Usage:
const product = await queries.getProduct(1);console.log(product?.details?.entries.name);console.log(product?.details?.entries.price);Lists (Arrays)
Section titled “Lists (Arrays)”SQL:
CREATE TABLE posts ( id INTEGER, tags VARCHAR[], scores INTEGER[]);TypeScript:
interface PostRow { id: number; tags: { items: (string | null)[] } | null; scores: { items: (number | null)[] } | null;}Usage:
const post = await queries.getPost(1);console.log(post?.tags?.items); // ['tech', 'news']console.log(post?.scores?.items); // [95, 87, 92]SQL:
CREATE TABLE settings ( id INTEGER, config MAP(VARCHAR, VARCHAR));TypeScript:
interface SettingsRow { id: number; config: { entries: { key: string; value: string }[] } | null;}Usage:
const settings = await queries.getSettings(1);for (const entry of settings?.config?.entries ?? []) { console.log(`${entry.key}: ${entry.value}`);}Nested Complex Types
Section titled “Nested Complex Types”DuckDB supports deeply nested types:
CREATE TABLE reports ( id INTEGER, data STRUCT( summary VARCHAR, metrics STRUCT(views INTEGER, clicks INTEGER)[], tags MAP(VARCHAR, VARCHAR[]) ));Type Mapping
Section titled “Type Mapping”| DuckDB Type | TypeScript Type |
|---|---|
INTEGER, BIGINT | number, bigint |
DOUBLE, FLOAT | number |
VARCHAR, TEXT | string |
BOOLEAN | boolean |
TIMESTAMP | { micros: bigint } |
DATE | { days: number } |
TIME | { micros: bigint } |
BLOB | { bytes: Uint8Array } |
UUID | { hugeint: bigint } |
DECIMAL(p,s) | { width: number; scale: number; value: bigint } |
STRUCT(...) | { entries: {...} } |
T[] | { items: T[] } |
MAP(K,V) | { entries: { key: K; value: V }[] } |
Async Pattern
Section titled “Async Pattern”All generated methods are async:
// Query methods return Promiseconst users: UserRow[] = await queries.allUsers();const user: UserRow | undefined = await queries.getUser(1);
// Exec methods return Promise<void>await queries.updateUser(1, 'Alice Smith');Connection Management
Section titled “Connection Management”import { DuckDBInstance } from '@duckdb/node-api';
// In-memory databaseconst instance = await DuckDBInstance.create(':memory:');
// File-based databaseconst instance = await DuckDBInstance.create('app.duckdb');
// Connectconst connection = await instance.connect();const queries = new MyApp(connection);
// Multiple connections (for concurrent access)const conn1 = await instance.connect();const conn2 = await instance.connect();
// Close connectionsconnection.closeSync();Bulk Inserts with Appenders
Section titled “Bulk Inserts with Appenders”DuckDB’s Appender API provides high-performance bulk inserts—significantly faster than individual INSERT statements. SQG generates type-safe appender wrappers using the TABLE annotation.
Defining an Appender
Section titled “Defining an Appender”Use the TABLE annotation with the :appender modifier:
-- MIGRATE 1CREATE TABLE events ( id INTEGER PRIMARY KEY, event_type VARCHAR NOT NULL, payload VARCHAR, timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
-- TABLE events :appenderGenerated Code
Section titled “Generated Code”SQG generates three components:
- Row interface - Type-safe structure for each row
- Appender class - Wrapper with
append(),appendMany(),flush(), andclose()methods - Factory method - Creates appender instances from your queries class
// ...
export interface EventsRow { id: number; event_type: string; payload: string | null; timestamp: DuckDBTimestampValue | null;}
/** Appender for bulk inserts into events */export class EventsAppender { constructor(public readonly appender: DuckDBAppender) {}
/** Append a single row */ append(row: EventsRow): this { this.appender.appendInteger(row.id); this.appender.appendVarchar(row.event_type); if (row.payload === null || row.payload === undefined) { this.appender.appendNull(); } else { this.appender.appendVarchar(row.payload); } if (row.timestamp === null || row.timestamp === undefined) { this.appender.appendNull(); } else { this.appender.appendTimestamp(row.timestamp); } this.appender.endRow(); return this; }
/** Append multiple rows */ appendMany(rows: EventsRow[]): this { for (const row of rows) { this.append(row); } return this; }
/** Flush buffered data to the table */ flush(): this { this.appender.flushSync(); return this; }
/** Flush and close the appender */ close(): void { this.appender.closeSync(); }}Basic Usage
Section titled “Basic Usage”// Create appenderconst appender = await queries.createEventsAppender();
// Append single rowappender.append({ id: 1, event_type: 'page_view', payload: '{"url": "/home"}', timestamp: null});
// Append multiple rowsappender.appendMany([ { id: 2, event_type: 'click', payload: null, timestamp: null }, { id: 3, event_type: 'scroll', payload: '{"depth": 50}', timestamp: null },]);
// Flush and closeappender.close();For large datasets, call flush() periodically to manage memory, then close() when done.
Type Mappings for Appenders
Section titled “Type Mappings for Appenders”Appender row interfaces use DuckDB’s native value types for temporal and binary data:
| DuckDB Column Type | TypeScript Appender Type |
|---|---|
INTEGER, SMALLINT, TINYINT | number |
BIGINT, HUGEINT | bigint |
DOUBLE, FLOAT | number |
VARCHAR, TEXT | string |
BOOLEAN | boolean |
DATE | DuckDBDateValue |
TIME | DuckDBTimeValue |
TIMESTAMP | DuckDBTimestampValue |
BLOB | DuckDBBlobValue |
UUID | string |
Note: Nullable columns use T | null types.
Working with Temporal Types
Section titled “Working with Temporal Types”For DATE, TIME, and TIMESTAMP columns, use DuckDB’s value constructors:
import { DuckDBDateValue, DuckDBTimeValue, DuckDBTimestampValue} from '@duckdb/node-api';
// Create timestamp valueconst timestamp = DuckDBTimestampValue.fromMicros(BigInt(Date.now()) * 1000n);
// Create date valueconst date = DuckDBDateValue.fromDays(Math.floor(Date.now() / 86400000));
appender.append({ id: 1, event_type: 'login', payload: null, timestamp: timestamp});Example: Analytics Query
Section titled “Example: Analytics Query”-- MIGRATE 1CREATE TABLE orders ( id INTEGER PRIMARY KEY, region VARCHAR NOT NULL, amount DECIMAL(10,2) NOT NULL, order_date DATE NOT NULL);
-- TESTDATA seedINSERT INTO orders (id, region, amount, order_date) VALUES (1, 'North', 100.50, DATE '2024-01-15'), (2, 'South', 250.75, DATE '2024-02-20'), (3, 'North', 175.25, DATE '2024-03-10'), (4, 'East', 300.00, DATE '2024-04-05'), (5, 'South', 125.50, DATE '2024-05-12');
-- QUERY sales_by_region@set start_date = 2024-01-01@set end_date = 2024-12-31SELECT region, COUNT(*) as order_count, SUM(amount) as total_sales, AVG(amount) as avg_order_valueFROM ordersWHERE order_date BETWEEN ${start_date} AND ${end_date}GROUP BY regionORDER BY total_sales DESC;// ...
async salesByRegion( start_date: string, end_date: string, ): Promise< { region: string | null; order_count: bigint | null; total_sales: { width: number; scale: number; value: bigint } | null; avg_order_value: number | null; }[] > { const sql = `SELECT region, COUNT(*) as order_count, SUM(amount) as total_sales, AVG(amount) as avg_order_valueFROM ordersWHERE order_date BETWEEN ? AND ?GROUP BY regionORDER BY total_sales DESC;`; const reader = await this.conn.runAndReadAll(sql, [start_date, end_date]); return reader.getRowObjects() as { region: string | null; order_count: bigint | null; total_sales: { width: number; scale: number; value: bigint } | null; avg_order_value: number | null; }[]; }}