SQG - Compile SQL to Type-Safe Code | Blog

Benchmarking DuckDB From Java: Fast INSERT, UPDATE, and DELETE

Tue, 21 Apr 2026 00:00:00 GMT

DuckDB’s Java documentation recommends the Appender as the efficient way to insert data. But two questions come up in practice: how do you efficiently UPDATE and DELETE? And is the Appender really the fastest option for INSERT, or are there faster alternatives?

We benchmarked 7 different methods — from naive row-by-row JDBC to the Appender, Arrow streams, and DuckDB’s Java table function API — across INSERT, UPDATE, and DELETE, scaling from 1K to 1M rows.

The Setup

We used a sensor-readings table with 9 columns covering DuckDB’s core types: UUID, TIMESTAMPTZ, DOUBLE, DECIMAL(10,2), SMALLINT, BOOLEAN, VARCHAR, and VARCHAR[] (same schema as our PostgreSQL benchmark).

Each method was warmed up, then measured multiple times (median taken). After every operation, we verified correctness: row counts, spot-checked values, and for UPDATE specifically verified the target column was actually modified. DuckDB runs on disk with a single connection.

All benchmark code is open source: examples/java-duckdb-benchmark.

INSERT Methods

We tested 6 insert methods:

1. Individual INSERT — One executeUpdate() per row in a transaction. Simple but slow: ~3.3K rows/sec at all sizes. (code)

2. JDBC Batch — Standard JDBC addBatch() / executeBatch() via the DuckDB JDBC driver. On PostgreSQL this is a solid optimization; on DuckDB it barely helps — ~5K rows/sec. (code)

3. Multi-value INSERT — Build INSERT INTO t VALUES (...),(...),(...) ... with up to 10K rows per statement. ~14K rows/sec. (code)

4. Appender — DuckDB’s recommended bulk insert API. Bypasses the SQL parser — you call beginRow(), append() for each column, and endRow(): (code)

try (var appender = ((DuckDBConnection) conn)
        .createAppender(DuckDBConnection.DEFAULT_SCHEMA, "sensor_readings")) {
    for (var r : rows) {
        appender.beginRow();
        appender.append(r.deviceId());
        appender.append(r.timestamp());
        appender.append(r.temperature());
        // ... remaining columns
        appender.endRow();
    }
}

5. Arrow Stream — Build Apache Arrow columnar vectors in Java, register them with DuckDB via registerArrowStream, then INSERT INTO ... SELECT FROM stream. Since Arrow vectors must be fully built before registering, the data is processed in chunks of 10K rows — each chunk requires a separate INSERT ... SELECT statement. (code)

6. Table Function (UDF) — Register a Java table function via DuckDB’s DuckDBFunctions.tableFunction() API. DuckDB pulls rows directly from a Java Iterator in chunks of 2,048. Unlike Arrow, this executes a single SQL statement regardless of data size — DuckDB calls the apply() callback repeatedly until the iterator is exhausted: (code)

DuckDBFunctions.tableFunction()
    .withName("_bench_insert_rows")
    .withFunction(new DuckDBTableFunction<>() {
        public long apply(DuckDBTableFunctionCallInfo info,
                          DuckDBDataChunkWriter output) {
            // Fill columnar vectors, up to output.capacity() rows
            // DuckDB calls this repeatedly until we return 0
        }
    })
    .register(conn);

// Then: INSERT INTO sensor_readings SELECT ... FROM _bench_insert_rows()

UPDATE and DELETE Methods

For UPDATE and DELETE, Individual and JDBC Batch work the same way as for INSERT — one statement per row or batch. (update code, delete code)

The interesting methods are the set-based approaches:

UNNEST list params — Pass all keys as parallel array parameters: (code)

DELETE FROM sensor_readings
WHERE (device_id, timestamp) IN (
    SELECT UNNEST(?::UUID[]), UNNEST(?::TIMESTAMPTZ[])
);

Arrow stream join — Stage keys as Arrow vectors, then join against the main table. Same registerArrowStream technique but with only the key columns. (update code, delete code)

DELETE FROM sensor_readings s
USING arrow_stream k
WHERE s.device_id = k.device_id::UUID AND s.timestamp = k.timestamp

Temp table + Appender — Create a temp table, bulk-load keys with the Appender, then execute a set-based join. This turns DuckDB’s INSERT-only Appender into a tool for bulk UPDATE and DELETE: (update code, delete code)

// 1. Create staging table
stmt.execute("CREATE TEMP TABLE _keys (device_id UUID, timestamp TIMESTAMPTZ)");

// 2. Bulk-load keys via Appender
try (var appender = ((DuckDBConnection) conn)
        .createAppender(DuckDBConnection.DEFAULT_SCHEMA, "_keys")) {
    for (var r : rows) {
        appender.beginRow();
        appender.append(r.deviceId());
        appender.append(r.timestamp());
        appender.endRow();
    }
}

// 3. Set-based delete
stmt.execute("""
    DELETE FROM sensor_readings s
    USING _keys k
    WHERE s.device_id = k.device_id AND s.timestamp = k.timestamp""");
stmt.execute("DROP TABLE _keys");

The same pattern works for UPDATE — add the new values to the staging table:

CREATE TEMP TABLE _update_keys (device_id UUID, timestamp TIMESTAMPTZ, temperature DOUBLE);
-- Appender-load keys + new values
UPDATE sensor_readings s
SET temperature = k.temperature
FROM _update_keys k
WHERE s.device_id = k.device_id AND s.timestamp = k.timestamp;

Table function join — Same as the insert table function, but DuckDB uses it as a key source in a DELETE ... USING or UPDATE ... FROM join. (update code, delete code)

The Results

DuckDB Insert / Update / Delete Benchmark

Comparing JDBC :batch, appender, Arrow, UNNEST list params, and temp-table join

INSERT: Throughput vs Total Rows

UPDATE: Throughput vs Total Rows

DELETE: Throughput vs Total Rows

INSERT results

1,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	345.9	2,891	29.3x slower
Batch (:batch)	216.8	4,613	18.3x slower
Multi-value INSERT	53.8	18,582	4.6x slower
Appender (:appender)	11.8	84,630	fastest
Arrow stream	19.2	51,977	1.6x slower
Table function	13.6	73,622	1.1x slower

5,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	1,582.3	3,160	64.4x slower
Batch (:batch)	1,009.2	4,954	41.1x slower
Multi-value INSERT	334.5	14,948	13.6x slower
Appender (:appender)	35.0	142,985	1.4x slower
Arrow stream	26.1	191,677	1.1x slower
Table function	24.6	203,537	fastest

10,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	3,016.6	3,315	111.4x slower
Batch (:batch)	1,950.2	5,128	72.0x slower
Multi-value INSERT	750.5	13,325	27.7x slower
Appender (:appender)	62.3	160,472	2.3x slower
Arrow stream	27.1	369,291	fastest
Table function	39.2	254,808	1.4x slower

50,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	15,062.9	3,319	106.5x slower
Batch (:batch)	9,805.0	5,099	69.3x slower
Multi-value INSERT	3,722.1	13,433	26.3x slower
Appender (:appender)	347.8	143,751	2.5x slower
Arrow stream	188.1	265,791	1.3x slower
Table function	141.5	353,377	fastest

100,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	30,299.7	3,300	128.7x slower
Batch (:batch)	19,854.5	5,037	84.3x slower
Multi-value INSERT	7,300.8	13,697	31.0x slower
Appender (:appender)	629.1	158,945	2.7x slower
Arrow stream	235.4	424,734	fastest
Table function	250.2	399,683	1.1x slower

500,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	-	-	skipped
Batch (:batch)	-	-	skipped
Multi-value INSERT	-	-	skipped
Appender (:appender)	3,120.3	160,243	2.6x slower
Arrow stream	1,288.7	387,994	1.1x slower
Table function	1,179.9	423,747	fastest

1,000,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	-	-	skipped
Batch (:batch)	-	-	skipped
Multi-value INSERT	-	-	skipped
Appender (:appender)	6,544.0	152,812	2.5x slower
Arrow stream	2,922.8	342,139	1.1x slower
Table function	2,608.4	383,377	fastest

UPDATE results

1,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	226.2	4,421	20.8x slower
Batch (:batch)	192.0	5,208	17.7x slower
UNNEST list params	12.2	81,756	1.1x slower
Arrow stream join	11.6	85,900	1.1x slower
Temp table join	10.9	91,933	fastest
Table function	11.0	90,759	~same

5,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	1,179.6	4,239	95.1x slower
Batch (:batch)	1,052.6	4,750	84.8x slower
UNNEST list params	17.0	294,131	1.4x slower
Arrow stream join	15.2	329,765	1.2x slower
Temp table join	12.4	402,927	fastest
Table function	12.6	396,662	~same

10,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	2,863.7	3,492	211.6x slower
Batch (:batch)	2,298.9	4,350	169.9x slower
UNNEST list params	24.0	417,146	1.8x slower
Arrow stream join	16.7	597,372	1.2x slower
Temp table join	13.5	739,041	fastest
Table function	15.1	660,487	1.1x slower

50,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	14,630.4	3,418	688.1x slower
Batch (:batch)	13,578.4	3,682	638.6x slower
UNNEST list params	119.9	417,188	5.6x slower
Arrow stream join	69.0	724,504	3.2x slower
Temp table join	21.3	2,351,675	fastest
Table function	24.6	2,033,870	1.2x slower

100,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	36,936.2	2,707	1199.7x slower
Batch (:batch)	31,714.7	3,153	1030.1x slower
UNNEST list params	232.0	431,070	7.5x slower
Arrow stream join	138.0	724,798	4.5x slower
Temp table join	30.8	3,248,036	fastest
Table function	36.9	2,707,682	1.2x slower

500,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	-	-	skipped
Batch (:batch)	-	-	skipped
UNNEST list params	1,223.8	408,575	7.6x slower
Arrow stream join	669.7	746,591	4.2x slower
Temp table join	160.9	3,106,897	fastest
Table function	198.9	2,513,792	1.2x slower

1,000,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual UPDATE	-	-	skipped
Batch (:batch)	-	-	skipped
UNNEST list params	2,340.3	427,299	9.1x slower
Arrow stream join	1,321.6	756,654	5.1x slower
Temp table join	257.3	3,887,168	fastest
Table function	330.8	3,023,035	1.3x slower

DELETE results

1,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	199.4	5,016	20.1x slower
Batch (:batch)	166.8	5,997	16.8x slower
UNNEST list params	10.9	91,474	1.1x slower
Arrow stream join	12.8	78,130	1.3x slower
Temp table join	10.6	94,494	1.1x slower
Table function	9.9	100,844	fastest

5,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	916.2	5,457	81.0x slower
Batch (:batch)	808.0	6,188	71.5x slower
UNNEST list params	15.6	320,613	1.4x slower
Arrow stream join	14.5	343,747	1.3x slower
Temp table join	11.8	423,315	~same
Table function	11.3	442,306	fastest

10,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	1,820.0	5,495	142.6x slower
Batch (:batch)	1,784.6	5,603	139.8x slower
UNNEST list params	22.5	444,311	1.8x slower
Arrow stream join	16.5	606,296	1.3x slower
Temp table join	12.8	783,441	fastest
Table function	14.5	689,856	1.1x slower

50,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	10,988.2	4,550	584.2x slower
Batch (:batch)	8,356.6	5,983	444.3x slower
UNNEST list params	94.6	528,407	5.0x slower
Arrow stream join	64.5	774,737	3.4x slower
Temp table join	18.8	2,658,112	fastest
Table function	25.1	1,993,260	1.3x slower

100,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	23,546.4	4,247	836.7x slower
Batch (:batch)	21,117.7	4,735	750.4x slower
UNNEST list params	192.2	520,213	6.8x slower
Arrow stream join	130.4	766,958	4.6x slower
Temp table join	28.1	3,553,297	fastest
Table function	40.0	2,500,696	1.4x slower

500,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	-	-	skipped
Batch (:batch)	-	-	skipped
UNNEST list params	956.8	522,595	9.4x slower
Arrow stream join	642.3	778,404	6.3x slower
Temp table join	101.6	4,922,569	fastest
Table function	148.8	3,359,951	1.5x slower

1,000,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual DELETE	-	-	skipped
Batch (:batch)	-	-	skipped
UNNEST list params	1,891.2	528,756	10.3x slower
Arrow stream join	1,248.1	801,192	6.8x slower
Temp table join	184.2	5,429,384	fastest
Table function	302.2	3,308,981	1.6x slower

System Information

Java	25 (Eclipse Adoptium)
OS	Linux 6.8.0-107-generic
Arch	amd64
CPUs	24
Max Memory	7,956 MB
DuckDB	v1.5.2
Warmup Runs	1
Measurement Runs	3

Appender: The Simple Default

The Appender is the right starting point for most Java applications. It’s simple, has no extra dependencies, and at 160K rows/sec it’s 30× faster than JDBC batch. For batches under 5K rows, it’s actually the fastest method.

Table Functions and Arrow: For Higher Throughput

Above 10K rows, Table Functions (427K rows/sec at 1M) and Arrow (381K rows/sec) both deliver 2.5–2.7× the Appender’s throughput:

Table Functions need no extra dependencies — just the DuckDB JDBC driver. You implement a bind()/init()/apply() callback that writes columnar vectors. The key advantage: the entire operation runs as a single SQL statement, regardless of data size. DuckDB streams through the callback internally. This is why Table Functions overtake Arrow at large sizes (500K+). For UPDATE/DELETE they place second behind temp-table-join at 3.0–3.4M rows/sec.
Arrow requires Apache Arrow JARs and more code (schema definitions, vector building, stream registration, off-heap memory management). Arrow vectors must be fully built in memory before registering, so data is processed in chunks — each chunk is a separate SQL statement. At 1M rows with 10K-row chunks, that’s 100 statements vs Table Function’s 1. Arrow wins at medium sizes (10K–100K) but falls behind Table Functions at 1M. It’s the right choice when your data is already columnar or when you need interop with other Arrow-based systems.

Temp Table + Appender: For Bulk Mutations

The temp-table-join pattern delivers 5.8M deletes/sec and 3.8M updates/sec at 1M rows — over 1,000× faster than JDBC batch. Instead of executing 1M individual statements, DuckDB executes a single set-based join. UNNEST (the other set-based approach) is 9–11× slower at 1M.

JDBC Batch: For Small Operations

On DuckDB, addBatch() / executeBatch() provides almost no speedup over individual statements (~5K vs ~3.3K rows/sec). DuckDB doesn’t rewrite batched statements the way PostgreSQL does with reWriteBatchedInserts. JDBC Batch is still the simplest option and works across both DuckDB and PostgreSQL, making it useful for portable code or small batches.

Methodology

DuckDB JDBC (in-process, on-disk database)
Java 25 (Temurin)
Apache Arrow 19.0.0
Each method: warmup + measurement runs, median reported
Data verified after every operation (row count + value checks)
All methods use a single connection
All methods accept Iterable (not List) — no method assumes the full dataset fits in memory
Methods that need batching (Arrow, UNNEST, Multi-value, JDBC Batch) chunk at 10,000 rows internally
The Table Function streams naturally via its Iterator-based callback at 2,048 rows per chunk

Try It Yourself

git clone https://github.com/sqg-dev/sqg
cd sqg/examples/java-duckdb-benchmark
just run            # normal (skips slow methods at large sizes)
just run --all      # include Individual/Batch at every size

How We Built This

The benchmark uses SQG, a type-safe SQL code generator. You write annotated .sql files with your queries, and SQG introspects them against a real database at build time to generate type-safe access code for Java, TypeScript, or Python. No ORM, no runtime reflection — just plain SQL in, type-safe code out.

For example, this annotation:

-- TABLE sensor_readings :appender

generates a complete type-safe SensorReadingsAppender class with append(UUID deviceId, OffsetDateTime timestamp, ...) — column types inferred directly from DuckDB.

From the single queries.sql file, SQG generated most of the code used in this benchmark:

The Appender for sensor_readings (from TABLE sensor_readings :appender)
Batch INSERT/UPDATE/DELETE methods (from :batch annotations)
UNNEST bulk delete/update methods (delete_readings_unnest)
The temp-table staging pattern: CREATE EXECs, staging Appenders (TABLE _delete_keys :appender), and set-based join EXECs
Individual row methods and verification queries

The Arrow and Table Function methods are the only hand-written ones — everything else is generated from SQL.

SQG can currently generate DuckDB appender code and also produce Arrow code for reading results (SELECT). We are planning to add more support for generating code for UDFs and methods which use Arrow as input.

Questions or feedback? Join the discussion on GitHub.

I Benchmarked 9 Ways to Insert Data Into PostgreSQL From Java. COPY BINARY Won by a Landslide.

Thu, 26 Mar 2026 00:00:00 GMT

TL;DR: We benchmarked 9 different methods of inserting data into PostgreSQL from Java, from naive individual INSERTs to DuckDB-via-Arrow pipelines. COPY BINARY wins decisively at scale: 712K rows/sec for 1M rows — 2x faster than COPY CSV, 5x faster than JDBC batch, and 18x faster than individual INSERTs. The sweet spot for most applications is reWriteBatchedInserts=true (one URL parameter, 2.5x speedup over naive batch). For high-volume ingestion, COPY BINARY via PgBulkInsert is unbeatable.

The Setup

We built a benchmark that simulates an IoT sensor ingestion pipeline — a table with 9 columns covering the interesting PostgreSQL types: UUID, TIMESTAMPTZ, DOUBLE PRECISION, NUMERIC, SMALLINT, BOOLEAN, TEXT, and TEXT[]. One million rows of randomized but deterministic sensor readings (temperature, humidity, pressure, battery level, anomaly flags, location tags).

Each method was warmed up (2 runs), then measured (5 runs, median taken). After every insert, we verified both row count and spot-checked the first and last 10 rows against source data. PostgreSQL 18 via TestContainers. Java 25.

All benchmark code is open source: examples/java-postgres-benchmark.

The Methods

Here’s what we tested, roughly in order of complexity:

1. Individual INSERT (the naive approach)

One executeUpdate() per row, wrapped in a single transaction. This is what most beginners write, and what many ORMs produce. (code)

2. Batch INSERT (`addBatch` / `executeBatch`)

Standard JDBC batching. Accumulate rows with addBatch(), flush with executeBatch(). The driver sends them as individual INSERT statements but pipelines the network calls. (code)

3. Batch INSERT with `reWriteBatchedInserts=true`

Same code as #2, but with one URL parameter added: ?reWriteBatchedInserts=true. The PostgreSQL JDBC driver automatically rewrites individual batched INSERTs into multi-value statements: INSERT INTO t VALUES (...),(...),.... This is the lowest-effort optimization you can make. (code)

4. Multi-value INSERT (hand-crafted)

Manually build INSERT INTO t VALUES (...),(...),(...) ... with chunks of 1,000 rows. The hand-rolled version of what reWriteBatchedInserts does automatically. (code)

5. UNNEST

INSERT INTO t SELECT * FROM unnest($1::uuid[], $2::timestamptz[], ...)

Pass one array per column instead of one parameter per cell. Only 9 parameters regardless of row count, so it avoids PostgreSQL’s 65K parameter limit. Reduces query planning overhead. (code, generated)

6. COPY CSV

PostgreSQL’s COPY ... FROM STDIN WITH (FORMAT CSV) via the JDBC driver’s CopyManager. Build a CSV string in memory, stream it to PostgreSQL. This is the standard “fast path” that most PostgreSQL performance guides recommend. (code)

7. COPY BINARY

PostgreSQL’s COPY ... FROM STDIN (FORMAT BINARY) via PgBulkInsert. Instead of text CSV, it writes PostgreSQL’s native binary wire format — no parsing overhead on the server side. (code, generated)

8. DuckDB → PostgreSQL

Load data into an in-memory DuckDB table via DuckDB’s Appender API (extremely fast — ~850K rows/sec into DuckDB), then push to PostgreSQL via DuckDB’s postgres extension: INSERT INTO pg.table SELECT * FROM local_table. DuckDB uses COPY BINARY under the hood for the transfer. (code, generated)

9. Arrow → DuckDB → PostgreSQL

Build Apache Arrow columnar vectors in Java, register them with DuckDB as a virtual table (zero-copy), then push to PostgreSQL via the postgres extension. Skips the DuckDB staging table entirely. (code)

The Results

The interactive report below shows all batch sizes from 100 to 1,000,000 rows:

PostgreSQL Insert Benchmark

Comparing 9 insert methods across batch sizes from 100 to 1,000,000 rows

Throughput vs Total Rows

100 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	9.0	11,057	10.0x slower
Batch INSERT	1.4	72,420	1.5x slower
Batch (rewrite=true)	1.4	69,433	1.6x slower
Multi-value INSERT	1.4	71,185	1.6x slower
UNNEST	1.3	74,810	1.5x slower
COPY CSV	0.9	110,549	fastest
COPY BINARY	2.0	51,234	2.2x slower
DuckDB -> PostgreSQL	3.0	33,531	3.3x slower
Arrow -> DuckDB -> PG	4.2	23,937	4.6x slower

1,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	39.0	25,612	9.7x slower
Batch INSERT	6.8	147,246	1.7x slower
Batch (rewrite=true)	5.1	196,576	1.3x slower
Multi-value INSERT	7.9	127,238	2.0x slower
UNNEST	6.4	155,451	1.6x slower
COPY CSV	4.0	249,664	fastest
COPY BINARY	4.9	205,923	1.2x slower
DuckDB -> PostgreSQL	7.5	133,547	1.9x slower
Arrow -> DuckDB -> PG	10.9	91,756	2.7x slower

10,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	266.0	37,589	16.5x slower
Batch INSERT	64.3	155,544	4.0x slower
Batch (rewrite=true)	36.1	276,876	2.2x slower
Multi-value INSERT	39.8	251,090	2.5x slower
UNNEST	40.5	247,087	2.5x slower
COPY CSV	32.2	310,479	2.0x slower
COPY BINARY	16.1	621,596	fastest
DuckDB -> PostgreSQL	40.4	247,228	2.5x slower
Arrow -> DuckDB -> PG	39.3	254,744	2.4x slower

100,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	2,650.2	37,733	17.8x slower
Batch INSERT	633.3	157,907	4.2x slower
Batch (rewrite=true)	375.6	266,216	2.5x slower
Multi-value INSERT	386.2	258,964	2.6x slower
UNNEST	392.0	255,124	2.6x slower
COPY CSV	278.0	359,671	1.9x slower
COPY BINARY	149.1	670,830	fastest
DuckDB -> PostgreSQL	354.2	282,339	2.4x slower
Arrow -> DuckDB -> PG	290.4	344,363	1.9x slower

1,000,000 rows

Method	Time (ms)	Rows/sec	vs Best
Individual INSERT	25,494.3	39,225	18.2x slower
Batch INSERT	6,173.2	161,990	4.4x slower
Batch (rewrite=true)	3,423.6	292,091	2.4x slower
Multi-value INSERT	3,960.3	252,504	2.8x slower
UNNEST	4,124.8	242,437	2.9x slower
COPY CSV	2,774.5	360,420	2.0x slower
COPY BINARY	1,403.7	712,418	fastest
DuckDB -> PostgreSQL	3,274.8	305,362	2.3x slower
Arrow -> DuckDB -> PG	2,830.8	353,263	2.0x slower

System Information

Java	Java 25 (Temurin 25+36-LTS)
OS	Linux 6.8.0-100-generic
Arch	amd64
CPUs	24
Max Memory	7,956 MB
PostgreSQL	18 (TestContainers)
Warmup Runs	2
Measurement Runs	5

How They Scale

The performance characteristics change with data volume. At small sizes (100-1,000 rows), the overhead of COPY setup makes simpler methods competitive. COPY CSV actually wins below ~5K rows. But once you cross 10K rows, COPY BINARY pulls away and the gap widens linearly.

At 1M rows, COPY BINARY delivers 712K rows/sec — 2x faster than COPY CSV, 2.4x faster than reWriteBatchedInserts, and 18x faster than individual INSERTs.

The DuckDB Detour: What We Learned

The DuckDB approach was the most fun to benchmark. DuckDB is known for being heavily optimized, and we were curious whether routing data through its postgres extension could beat a direct COPY. The idea: spin up an in-memory DuckDB instance as a temporary buffer — no files, no persistence, just a fast columnar engine to stage data before pushing it to PostgreSQL.

DuckDB Appender path (method 8): DuckDB’s appender loads 1M rows into an in-memory table in ~1.1s (~935K rows/sec). But then the DuckDB postgres extension takes ~2.3s to push that data to PostgreSQL. Total: 3.3s. The bottleneck is the cross-engine transfer.

Arrow path (method 9): Building Arrow vectors directly and registering them as a virtual table with DuckDB (zero-copy, no staging table needed) is faster than the appender path (~635ms vs ~1.1s for 1M rows). DuckDB scans the Arrow memory directly. But the copy-to-PG step takes the same ~2.2s. Total: 2.8s.

Key insight: DuckDB’s postgres extension already uses COPY BINARY under the hood (we confirmed this by reading the source). So the DuckDB → PG transfer is doing the same thing as PgBulkInsert, but with additional overhead from the cross-engine bridge. Direct PgBulkInsert cuts out the middleman.

Where DuckDB shines is if your data is already in DuckDB (from a Parquet file, a CSV, a data pipeline). In that case, ATTACH + INSERT INTO is the most natural path and gives you COPY BINARY for free without any extra libraries. For data that starts in Java, going direct is faster.

Recommendations

For most applications: `reWriteBatchedInserts=true`

If you’re using JDBC (directly or through Spring/Hibernate), just add ?reWriteBatchedInserts=true to your connection URL. One parameter change, zero code changes, ~2.5x throughput improvement over naive batching. This is the best ROI optimization.

jdbc:postgresql://host:5432/db?reWriteBatchedInserts=true

For bulk ingestion: COPY BINARY via PgBulkInsert

If you’re ingesting thousands or millions of rows — ETL pipelines, IoT data, analytics ingestion, data migrations — use PgBulkInsert by Philipp Wagner. It’s a well-maintained Java library that handles the binary wire format, supports all PostgreSQL types (including arrays, UUIDs, JSONB, timestamps with timezone), and delivers ~700K rows/sec.

For COPY CSV users

If you’re already using COPY CSV (a common recommendation), know that switching to COPY BINARY roughly doubles your throughput. The difference is server-side parsing: CSV requires text → type conversion for every value, while binary sends data in PostgreSQL’s native format.

What about UNNEST?

UNNEST (INSERT INTO t SELECT * FROM unnest(...)) is often recommended as “faster than multi-value INSERT” because it reduces query planning overhead. In our benchmark with 9 columns, it was roughly equivalent to multi-value INSERT at most sizes. The advantage may be more pronounced with wider tables or simpler column types. Its real benefit is avoiding the 65K parameter limit — with 9 columns, multi-value INSERT maxes out at ~7,200 rows per statement.

How We Built This

The benchmark was built using SQG, a SQL-first code generator. SQG reads annotated .sql files and generates type-safe database access code for Java, TypeScript, and Python — queries, inserts, COPY BINARY appenders, DuckDB bulk insert appenders, all from a single SQL file.

In this benchmark, SQG generated the COPY BINARY appender (with PgBulkInsert mapper and row record), the DuckDB appender, the UNNEST insert, the individual INSERT, and the verification queries — all from queries.sql. The only hand-written insert methods are Batch INSERT and COPY CSV, which use raw JDBC.

Methodology

PostgreSQL 18 (alpine) via TestContainers 2.0.4
Java 25 (Temurin 25+36-LTS)
DuckDB 1.5.1, Apache Arrow 19.0.0, PgBulkInsert 9.0.0
Each method: 2 warmup runs + 5 measurement runs, median reported
Data verified after every insert (row count + first/last row spot checks)
All methods use a single connection (no parallel streams)
TestContainers runs PostgreSQL in Docker with default settings (no tuning)

Try It Yourself

git clone https://github.com/sqg-dev/sqg
cd sqg/examples/java-postgres-benchmark
just run

This generates the code from SQL files, runs the full benchmark, and writes an HTML report with interactive charts.

Credits

PgBulkInsert by Philipp Wagner — the library that makes COPY BINARY accessible from Java
DuckDB and its postgres extension
TestContainers

Have questions or want to challenge the methodology? Open an issue on GitHub.

SQG v0.10.0: Java Streams & List Type Support

Thu, 05 Mar 2026 00:00:00 GMT

SQG is a type-safe SQL code generator — you write .sql files with annotated queries, and it generates strongly-typed database access code for TypeScript and Java by introspecting your queries against real database engines at build time.

Here’s what’s new in v0.10.0.

Java: Stream-based result methods

Generated Java code now includes methods that return Stream<T> in addition to List<T>. This gives you lazy evaluation, easier composition with the standard library, and avoids materializing large result sets into memory when you don’t need to.

try (Stream<User> users = queries.getAllUsersStream()) {
    users.forEach(user -> process(user));
}

The stream holds a reference to the underlying ResultSet, so it needs to be closed after use — hence the try-with-resources.

Java: Better array/list field support

Array columns like TEXT[] or INTEGER[] are now handled correctly in generated Java code. Previously these types could produce incorrect mappings — they now resolve to proper List<String>, List<Integer>, etc.

DuckDB: List types in Appender

The DuckDB appender now supports list/array column types. If your table has a column like tags VARCHAR[], the generated appender method accepts the corresponding list type and writes it correctly using DuckDB’s bulk insert API.

Upgrade with npm install -g @sqg/sqg@0.10.0 or update your project’s dependency. Full source on GitHub. Try it out in the playground.

Discuss this on Hackernews.

PostgreSQL Support & Built-in Migration Tracking

Fri, 06 Feb 2026 00:00:00 GMT

SQG v0.8.0 brings two major features: improved PostgreSQL support for Java and a built-in migration runner that tracks which migrations have been applied.

Built-in Migration Tracking

Until now, SQG generated a getMigrations() method that returned raw SQL strings — you were responsible for tracking which ones had been applied. That meant writing your own migration runner or integrating an external tool.

With v0.8.0, you can now enable a built-in applyMigrations() method:

sqg.yaml

version: 1
name: my-app

sql:
  - files:
      - queries.sql
    gen:
      - generator: typescript/sqlite
        output: ./src/generated/
        config:
          migrations: true  # enable built-in migration tracking

Then in your application:

import Database from 'better-sqlite3';
import { MyApp } from './generated/my-app';

const db = new Database('app.db');

// One line — creates tracking table, checks what's applied, runs the rest
MyApp.applyMigrations(db);

const queries = new MyApp(db);

How It Works

The generated applyMigrations() method:

Creates a _sqg_migrations table if it doesn’t exist
Checks which migrations have already been applied for this project
Applies new migrations in order
Records each migration with a timestamp
Wraps everything in a transaction for safety

The tracking table uses a composite primary key of (project, migration_id), which means multiple SQG projects can share the same database without conflicts. The project name comes from the name field in your sqg.yaml.

// Default project name from sqg.yaml
MyApp.applyMigrations(db);

// Override for multi-tenant scenarios
MyApp.applyMigrations(db, 'tenant-123');

Supported Across All Generators

The migration runner works with every SQG generator:

Generator	Method
TypeScript/SQLite (better-sqlite3)	`MyApp.applyMigrations(db)`
TypeScript/SQLite (node:sqlite)	`MyApp.applyMigrations(db)`
TypeScript/SQLite (libSQL)	`await MyApp.applyMigrations(client)`
TypeScript/DuckDB	`await MyApp.applyMigrations(conn)`
Java/JDBC (any engine)	`MyApp.applyMigrations(connection)`
Java/DuckDB Arrow	`Analytics.applyMigrations(connection)`

Each implementation uses the appropriate transaction mechanism for its engine — BEGIN IMMEDIATE for SQLite, setAutoCommit(false) for JDBC, and so on.

Still Optional

The feature is entirely opt-in. Without config.migrations: true, SQG generates the same getMigrations() method as before. You can continue using external migration tools like Flyway, Liquibase, or your own scripts.

PostgreSQL Improvements

v0.8.0 significantly improves PostgreSQL support for the java/postgres generator.

User-Defined Types (ENUMs)

SQG now introspects PostgreSQL’s pg_type system catalog to resolve user-defined types. This means ENUMs work out of the box:

-- MIGRATE 1
CREATE TYPE task_status AS ENUM ('pending', 'active', 'completed', 'cancelled');

CREATE TABLE tasks (
    id SERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    status task_status DEFAULT 'pending'
);

-- QUERY get_tasks_by_status
@set status = 'active'
SELECT id, title, status FROM tasks WHERE status = ${status}::task_status;

SQG resolves the ENUM OID from pg_type and generates a type-safe Java enum class with getValue() and fromValue() methods. Query parameters and results use the generated enum type directly instead of raw strings. See the Java + JDBC PostgreSQL documentation for details.

Array Types

PostgreSQL array columns like TEXT[] and INTEGER[] are now properly mapped to List<T> in Java:

CREATE TABLE tasks (
    id SERIAL PRIMARY KEY,
    tags TEXT[],
    priority_scores INTEGER[]
);

// Generated record
record GetAllTasksResult(Integer id, String title, List<String> tags, List<Integer> priorityScores) {}

for (var task : queries.getAllTasks()) {
    System.out.println(task.tags());            // [urgent, backend]
    System.out.println(task.priorityScores());  // [10, 20, 30]
}

TIMESTAMPTZ Support

TIMESTAMPTZ columns are now correctly mapped to OffsetDateTime (instead of LocalDateTime), preserving timezone information:

// TIMESTAMPTZ -> OffsetDateTime with UTC offset
record EventResult(Integer id, OffsetDateTime createdAt) {}

Automatic Testcontainers

If the SQG_POSTGRES_URL environment variable is not set, SQG now automatically starts a PostgreSQL container using Testcontainers. This makes it easy to get started without installing PostgreSQL locally — just have Docker running:

# No env var needed — SQG starts a container automatically
sqg sqg.yaml

For CI/CD or production builds, set the environment variable to point to your PostgreSQL server:

export SQG_POSTGRES_URL="postgresql://user:password@localhost:5432/mydb"
sqg sqg.yaml

Get Started

Install or update SQG:

pnpm add -g @sqg/sqg@latest

Enable migration tracking by adding config.migrations: true to your generator config. Check out the updated SQL Syntax Reference and generator documentation for full details.

Have questions or feedback? Open an issue on GitHub.

SQLite Driver Benchmark: Comparing better-sqlite3, node:sqlite, libSQL, Turso

Mon, 19 Jan 2026 00:00:00 GMT

Choosing the right SQLite driver for Node.js can impact your application’s performance.

We ran performance tests comparing better-sqlite3, node:sqlite, libSQL, and Turso across common database operations. Here’s what we found.

The Four SQLite Options for Node.js

better-sqlite3

This library has been tested in production for years, offering excellent performance through a synchronous API. The synchronous API makes it very performant and easy to use, and is perfect for fast queries.

→ GitHub better-sqlite3

node:sqlite

Node.js 22+ includes a built-in SQLite module (still experimental). It provides zero-dependency SQLite access with a synchronous API similar to better-sqlite3, and also an asynchronous API.

→ Node.js SQlite Documentation

libSQL

An open-source fork of SQLite created by the Turso team. It provides an async API and supports both local database files and remote libSQL/Turso servers. Fully compatible with the original SQLite (file format, API)

→ libSQL TypeScript

Turso

Turso is a SQLite compatible database written in Rust. Currently still in beta.

→ Turso for Javascript

Benchmark Methodology

We tested all four drivers with identical queries. A simple scenario with a user and posts table.

10,000 users and 500,000 posts (50 per user)
Optimized pragma settings (WAL mode, 64MB cache, memory-mapped I/O)
Various query patterns: simple selects, indexed lookups, JOINs, aggregates, inserts, updates

Note: better-sqlite3 and node:sqlite use synchronous APIs, while libSQL and Turso use async/await.

SQLite Configuration

All databases used these optimized settings:

PRAGMA journal_mode = WAL;       -- Write-Ahead Logging
PRAGMA synchronous = NORMAL;      -- Balance safety/speed
PRAGMA cache_size = -64000;       -- 64MB cache
PRAGMA temp_store = MEMORY;       -- Temp tables in memory
PRAGMA mmap_size = 268435456;     -- 256MB memory-mapped I/O

Results

SQLite Driver Benchmark Report

Comparing better-sqlite3 vs node:sqlite vs libsql vs turso (baseline: node:sqlite)

System Information

Node.js Version	v25.3.0
Platform	linux
Architecture	x64
CPU	12th Gen Intel(R) Core(TM) i9-12900K
CPU Cores	24
Total Memory	31.07 GB

Summary

Operation	better-sqlite3	node:sqlite	libsql	turso	Winner	vs node:sqlite
getAllUsers	360 ops/s	268 ops/s	50 ops/s	104 ops/s	better-sqlite3	1.34x
getUserById	1,223,260 ops/s	1,073,001 ops/s	61,093 ops/s	707,859 ops/s	better-sqlite3	1.14x
getUserByEmail	557,631 ops/s	457,659 ops/s	49,510 ops/s	233,913 ops/s	better-sqlite3	1.22x
countUsers (pluck)	538,031 ops/s	398,431 ops/s	108,632 ops/s	5,593 ops/s	better-sqlite3	1.35x
getPostsByUser	1,090,293 ops/s	980,550 ops/s	47,304 ops/s	414,672 ops/s	better-sqlite3	1.11x
getPublishedPosts (JOIN)	27 ops/s	27 ops/s	54 ops/s	7 ops/s	libsql	1.98x
getPostWithAuthor (JOIN :one)	477,271 ops/s	379,911 ops/s	32,433 ops/s	236,297 ops/s	better-sqlite3	1.26x
countPostsByUser (pluck)	1,151,783 ops/s	689,478 ops/s	111,824 ops/s	377,235 ops/s	better-sqlite3	1.67x
insertUser	53,693 ops/s	41,291 ops/s	28,385 ops/s	63,017 ops/s	turso	1.53x
updatePostViews	136,399 ops/s	97,956 ops/s	53,598 ops/s	59,273 ops/s	better-sqlite3	1.39x

Key Findings

better-sqlite3 is the fastest for most operations, with node:sqlite second
Turso has a surprisingly slow query for countPostsByUser (better-sqlite3 is almost 100x faster here). I did not investigate why this is, it might be that the Turso database like many others (eg Postgresql) needs to scan the full table in order to count the number of rows and has no fast handling for this special case.

Benchmark Implementation

The benchmark is implemented using SQG, a SQL to code generator.

One advantage of using a code generator like SQG is that you can switch between SQLite drivers without rewriting your queries. SQG generates type-safe code for all four drivers from the same SQL file:

# sqg.yaml - generate code for multiple drivers
sql:
  - files:
      - queries.sql
    gen:
      - generator: typescript/sqlite/better-sqlite3
        output: ./src/db-better-sqlite3.ts
      - generator: typescript/sqlite/node
        output: ./src/db-node-sqlite.ts
      - generator: typescript/sqlite/libsql
        output: ./src/db-libsql.ts
      - generator: typescript/sqlite/turso
        output: ./src/db-turso.ts

This makes it easy to benchmark with your actual queries and switch drivers by changing imports.

Try It Yourself

The benchmark code is available in our examples repository:

git clone https://github.com/sqg-dev/sqg
cd sqg/examples/typescript-sqlite-benchmark
pnpm install
pnpm generate
pnpm bench

This will generate an HTML report with the results table shown above.

Conclusion

I think for most applications, better-sqlite3 remains the best choice.

Benchmarks generated using SQG. Have questions? Open an issue on GitHub.

Introducing SQG

Wed, 17 Dec 2025 00:00:00 GMT

We’re excited to announce SQG (SQL Query Generator), a tool that generates type-safe database access code from your SQL queries.

The Problem

Writing database code means maintaining SQL queries and matching TypeScript/Java types. When schemas change, you update both—and hope nothing breaks. ORMs abstract SQL away, but what if you want to write SQL directly while keeping type safety?

The Solution

SQG reads annotated .sql files, executes queries against real databases to introspect column types, and generates fully-typed wrapper code.

Write your SQL with simple annotations:

-- MIGRATE createUsersTable
CREATE TABLE users (id INTEGER PRIMARY KEY,
                    name TEXT NOT NULL,
                    email TEXT);

-- QUERY getUserById :one
@set id = 1
SELECT id, name, email FROM users WHERE id = ${id};

-- QUERY getUsers
SELECT id, name, email FROM users;

-- EXEC insertUser
@set name = 'John'
@set email = 'john@example.com'
INSERT INTO users (name, email) VALUES (${name}, ${email});

SQG generates type-safe code:

export class Queries {
  getUserById(id: number): { id: number; name: string; email: string | null } | undefined
  getUsers(): { id: number; name: string; email: string | null }[]
  insertUser(name: string, email: string): RunResult
}

Key Features

Type-safe by design - Column types inferred from your actual database
Multiple databases - SQLite, DuckDB, and PostgreSQL
Multiple languages - Generate TypeScript or Java from the same SQL
DBeaver compatible - Develop queries in DBeaver, generate code from the same file
Zero runtime overhead - Generated code is plain function calls

Get Started

Install SQG:

pnpm add -g @sqg/sqg
pnpm approve-builds -g  # needed for sqlite dependency

Create a sqg.yaml config and your SQL file, then run sqg sqg.yaml. Check out the Getting Started guide for details.

We’d love your feedback! File issues and feature requests on GitHub.

SQG - Compile SQL to Type-Safe Code | Blog

Benchmarking DuckDB From Java: Fast INSERT, UPDATE, and DELETE

The Setup

INSERT Methods

UPDATE and DELETE Methods

The Results

DuckDB Insert / Update / Delete Benchmark

INSERT: Throughput vs Total Rows

UPDATE: Throughput vs Total Rows

DELETE: Throughput vs Total Rows

1,000 rows

5,000 rows

10,000 rows

50,000 rows

100,000 rows

500,000 rows

1,000,000 rows

1,000 rows

5,000 rows

10,000 rows

50,000 rows

100,000 rows

500,000 rows

1,000,000 rows

1,000 rows

5,000 rows

10,000 rows

50,000 rows

100,000 rows

500,000 rows

1,000,000 rows

System Information

Appender: The Simple Default

Table Functions and Arrow: For Higher Throughput

Temp Table + Appender: For Bulk Mutations

JDBC Batch: For Small Operations

Methodology

Try It Yourself

How We Built This

I Benchmarked 9 Ways to Insert Data Into PostgreSQL From Java. COPY BINARY Won by a Landslide.

The Setup

The Methods

1. Individual INSERT (the naive approach)

2. Batch INSERT (addBatch / executeBatch)

3. Batch INSERT with reWriteBatchedInserts=true

4. Multi-value INSERT (hand-crafted)

5. UNNEST

6. COPY CSV

7. COPY BINARY

8. DuckDB → PostgreSQL

9. Arrow → DuckDB → PostgreSQL

The Results

PostgreSQL Insert Benchmark

Throughput vs Total Rows

100 rows

1,000 rows

10,000 rows

100,000 rows

1,000,000 rows

System Information

How They Scale

The DuckDB Detour: What We Learned

Recommendations

For most applications: reWriteBatchedInserts=true

For bulk ingestion: COPY BINARY via PgBulkInsert

For COPY CSV users

What about UNNEST?

How We Built This

Methodology

Try It Yourself

Credits

SQG v0.10.0: Java Streams & List Type Support

Java: Stream-based result methods

Java: Better array/list field support

DuckDB: List types in Appender

PostgreSQL Support & Built-in Migration Tracking

Built-in Migration Tracking

How It Works

Supported Across All Generators

Still Optional

2. Batch INSERT (`addBatch` / `executeBatch`)

3. Batch INSERT with `reWriteBatchedInserts=true`

For most applications: `reWriteBatchedInserts=true`