Skip to content

feat(jdbc-driver): jdbc:vortex: driver to query Vortex files from any JDBC tool - #568

Open
dfa1 wants to merge 7 commits into
mainfrom
feat/jdbc-driver-566
Open

dfa1 wants to merge 7 commits into
mainfrom
feat/jdbc-driver-566

Conversation

@dfa1

@dfa1 dfa1 commented Oct 18, 2026

Copy link
Copy Markdown
Owner

Closes #566.

What

  • VortexCalciteOptions + VortexCalcite.connect(Path): a directory of .vortex files becomes SQL tables named after the files; columns come from each file's dtype.
  • VortexSchemaFactory: the same, from a jdbc:calcite:model=... URL (operand: directory, files; other keys rejected).
  • New module jdbc-driver (vortex-jdbc-driver): jdbc:vortex:<dir>?schema=..&table.<name>=<path>, registered via META-INF/services/java.sql.Driver, shipped as a self-contained -all jar (28.7 MB) for IntelliJ/DBeaver.
  • Fix: VortexCalcite.connect used DriverManager, which fails with "No suitable driver" when the jar is loaded by its own classloader (separate commit).

Notes

  • Non-breaking: connect(String, Map) and new VortexSchema(Map) delegate to the options.
  • ShadedJarIT (failsafe, after package) loads only the shaded jar in an isolated classloader: checks the merged java.sql.Driver entries, no signature files, and runs a query. It is what found the DriverManager bug.
  • Calcite dependencies excluded from the driver jar were each tested by running the calcite module's whole SQL suite without them; httpcore5, json-path, JTS and Jackson cannot go (planner init). Re-check on Calcite upgrades.
  • Model URLs (jdbc:calcite:model=) are not available through the driver jar (YAML excluded); use jdbc:vortex:.
  • No vortex-jni cross-check test: nothing here changes what is read or decoded, only how files are listed and named.

Not done

🤖 Generated with Claude Code

dfa1 and others added 5 commits October 18, 2026 20:27
VortexCalciteOptions (schema name, directory, explicit tables) replaces
passing a raw table map; connect(Path) is the one-call shortcut. The
columns still come from each file's dtype. Existing connect(String, Map)
and new VortexSchema(Map) delegate to the options, so nothing breaks.

Part of #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…odel

Lets a jdbc:calcite:model=... URL, as an IDE data source holds it, expose a
directory of .vortex files. operand takes directory and files; any other key
is rejected so a typo cannot silently expose no tables. The key is files, not
tables: Calcite fills operand.tables with the model's own empty list.

A model cannot set connection properties, so lex=JAVA must be in the URL.

Part of #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…r works

VortexCalcite.connect called DriverManager.getConnection("jdbc:calcite:"),
which only finds drivers already registered. A jar loaded by its own
classloader (an IDE data source, an application server) never had Calcite's
driver registered, so connect failed with "No suitable driver". Instantiate
Calcite's driver directly.

Found by the shaded-jar integration test added with the jdbc-driver module.

Part of #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…L tools

New module vortex-jdbc-driver: a java.sql.Driver registered through
META-INF/services that parses jdbc:vortex:<dir>?schema=..&table.<name>=<path>
into VortexCalciteOptions and connects through VortexCalcite. Unknown keys
fail with SQLState 08001 so a typo cannot silently expose no tables.

The -all shaded jar merges the java.sql.Driver service entries, strips
signature files and bundles the zstd natives. ShadedJarIT loads it alone in
an isolated classloader after package and runs a query.

Closes #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eaches

Drops the Avatica HTTP client, YAML, sketches, Hilbert-curve and connection
pooling jars: the shaded jar goes from 32.4 to 28.7 MB. Each was excluded on
the calcite module's classpath and its whole SQL suite re-run; httpcore5,
json-path and JTS cannot go, the planner initializes with them.

Part of #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dfa1 and others added 2 commits October 18, 2026 22:34
Setup, URL options, query tips (back-tick quoting, what is cheap), the
out-of-memory case for queries that hold every row in IntelliJ's JDBC
process, and a troubleshooting table. From a real IntelliJ session.

Part of #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
It is a bug to fix, not behavior to document.

Part of #566.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(jdbc): jdbc:vortex: driver to query Vortex files from any JDBC tool

1 participant