Fuzzing for #493: every target, a FlatBuffer verifier, and the bugs they found

.github/workflows/fuzz.yml

@@ -0,0 +1,77 @@
+name: Fuzz
+
+# Weekly coverage-guided fuzzing (ADR 0020, #493). Every @FuzzTest replays its saved inputs in the
+# ordinary build; this job explores new ones, one target per matrix job because Jazzer fuzzes a
+# single target per JVM. Scheduled, not on the PR path: a run is unbounded exploration, so a
+# failure is a finding to triage (the crash input is uploaded; commit it next to the target's
+# saved inputs with the fix), not a broken build.
+
+on:
+  schedule:
+    - cron: '0 3 * * 6'   # Saturdays 03:00 UTC
+  workflow_dispatch:
+    inputs:
+      duration:
+        description: 'Fuzzing time per target (e.g. 10m, 1h)'
+        default: '30m'
+
+jobs:
+  fuzz:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - { module: fuzz, target: 'VortexReaderFuzzTest#openMalformedFile' }
+          - { module: fuzz, target: 'VortexReaderFuzzTest#scanMalformedFile' }
+          - { module: fuzz, target: 'VortexReaderFuzzTest#filterMalformedFile' }
+          - { module: fuzz, target: 'VortexReaderFuzzTest#parseMetadataBlobs' }
+          - { module: fuzz, target: 'FsstFuzzTest#compressDecompressRoundTrip' }
+          - { module: integration, target: 'WriteReadFuzzIntegrationTest#javaRoundTrip' }
+          - { module: integration, target: 'WriteReadFuzzIntegrationTest#jniReadsJavaWrite' }
+    name: ${{ matrix.target }}
+
+    steps:
+      - uses: actions/checkout@v7
+
+      - name: Set up Azul Zulu JDK 25
+        uses: actions/setup-java@v6.0.1
+        with:
+          distribution: zulu
+          java-version: '25'
+
+      - name: Cache Maven repository
+        uses: actions/cache@v6
+        with:
+          path: ~/.m2/repository
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Fuzz
+        env:
+          JAZZER_FUZZ: '1'
+          JAZZER_MAX_DURATION: ${{ inputs.duration || '30m' }}
+        run: |
+          if [ "${{ matrix.module }}" = integration ]; then
+            ./mvnw -B verify -pl integration -am -Dit.test='${{ matrix.target }}' \
+                -Dtest=none -Dsurefire.failIfNoSpecifiedTests=false
+          else
+            ./mvnw -B test -pl fuzz -am -Dtest='${{ matrix.target }}' \
+                -Dsurefire.failIfNoSpecifiedTests=false
+          fi
+
+      # Jazzer writes a finding next to the target's saved inputs; the committed crash-* inputs
+      # are regressions already fixed, so only the new, untracked ones are collected.
+      - name: Collect the new crash input
+        if: failure()
+        run: |
+          mkdir -p findings
+          git ls-files --others --exclude-standard | grep '/crash-' | xargs -r -I{} cp --parents {} findings/
+
+      - name: Upload the crash input
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: crash-${{ strategy.job-index }}
+          path: findings/

CHANGELOG.md

@@ -18,6 +18,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Custom extension types: register an `ExtensionDecoder` on `ReadRegistry` and an `ExtensionEncoder` on `WriteRegistry`, with a `validate` hook on both sides, and `Chunk.as` decodes any registered type. ([#530](https://github.com/dfa1/vortex-java/issues/530))
 
 ### Changed
+- **Breaking:** `VortexWriter.create` refuses a nullable root struct: Rust reads a nullable struct only with a validity child the writer never wrote, so such files were unreadable outside vortex-java. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- **Breaking:** a file that does not embed its dtype is refused when it opens, as Rust refuses it unless the caller supplies the dtype, instead of failing the scan with `NullPointerException`. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- **Breaking:** every FlatBuffer in a file is verified before it is read, as Rust verifies it, so corrupt file metadata fails with `VortexException`, and like Rust the reader now refuses tables nested more than 64 deep. ([#493](https://github.com/dfa1/vortex-java/issues/493))
 - SQL `SUM` and `AVG` of a decimal column, grouped or not, are 4 to 6 times faster: they are totalled exactly as unscaled longs in the scan instead of by Calcite with a `BigDecimal` per row. ([#563](https://github.com/dfa1/vortex-java/pull/563))
 - SQL `COUNT(DISTINCT col)` is up to 4 times faster: distinct values are counted in the scan, integers by their bits in a primitive map, instead of Calcite hashing a boxed value per row. ([#562](https://github.com/dfa1/vortex-java/pull/562))
 - A SQL `WHERE` comparing a `SMALLINT` or `TINYINT` column with a literal is pushed into the scan and its aggregates, instead of being left to Calcite because of the cast Calcite adds. ([#560](https://github.com/dfa1/vortex-java/pull/560))
@@ -41,6 +44,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Breaking:** `ExtensionId` is a sealed `WellKnown`/`Custom` type like `EncodingId`, so third-party extension ids can be named, and `ExtensionId.parse` returns the id instead of an `Optional`. ([#530](https://github.com/dfa1/vortex-java/issues/530))
 
 ### Fixed
+- A file whose array FlatBuffer holds a corrupt offset fails the read with `VortexException` instead of `IndexOutOfBoundsException`. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- A column whose dtype does not fit its bit-packed, delta or RLE encoding fails with `VortexException` instead of `ClassCastException`, and an empty RLE boolean chunk reads. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- A struct dtype missing its field names or types, or a struct layout whose children do not match its fields, is rejected when the file opens, as Rust does, instead of failing the scan with `ArrayIndexOutOfBoundsException`. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- An empty `vortex.runend` array reads instead of being refused as an empty ends buffer. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- A `vortex.zstd_buffers` array whose wrapped encoding id holds a control character fails with `VortexException` instead of `IllegalArgumentException`. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- A constant `u8`, `u16` or `u32` column past the signed range of its width is written so vortex-jni reads it; its value was written sign-extended, which Rust refuses. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- `vortex.sequence` is written only where Rust writes it: not for empty or constant data, nor for a step that overflows the column type (a decreasing unsigned column, an `i8` step of 255), and unsigned values past the signed range are written so vortex-jni reads them. ([#493](https://github.com/dfa1/vortex-java/issues/493))
+- A file whose encoded child holds fewer values than its array needs, other than the single value of a constant child, fails with `VortexException` instead of dividing by zero or silently repeating values. ([#493](https://github.com/dfa1/vortex-java/issues/493))
 - SQL `WHERE` on an `I16`, `U8` or `I8` column no longer fails with a `ClassCastException`. ([#556](https://github.com/dfa1/vortex-java/pull/556))
 - SQL `COUNT(DISTINCT col)` and aggregates with a `FILTER (WHERE ...)` clause return the right answer; the statistics push-down returned the row count and ignored the filter. ([#554](https://github.com/dfa1/vortex-java/pull/554))
 - SQL `MIN`/`MAX` of a string column whose values fall outside Latin-1 answers instead of failing the query. ([#554](https://github.com/dfa1/vortex-java/pull/554))

CLAUDE.md

@@ -84,8 +84,9 @@ and merge commits are disabled): squash a PR's commits locally into one, force-p
 scripts/simd-benchmark.sh                  # auto-vectorized vs Vector API kernels, table of speedups (also a CI job)
 scripts/hydrate-raincloud-corpus.sh --max-mb 200   # hydrate real-world conformance corpus (#205), then:
 ./mvnw verify -pl integration -am -Dvortex.it.excludedGroups= -Dit.test="RaincloudConformanceIntegrationTest"
-./mvnw test -pl fuzz -am -Dvortex.fuzz.excludedGroups=                # Jazzer regression mode (ADR 0020)
-JAZZER_FUZZ=1 ./mvnw test -pl fuzz -am -Dvortex.fuzz.excludedGroups=  # actually fuzz (unbounded)
+JAZZER_FUZZ=1 JAZZER_MAX_DURATION=5m ./mvnw test -pl fuzz -am -Dtest='VortexReaderFuzzTest#scanMalformedFile' -Dsurefire.failIfNoSpecifiedTests=false  # fuzz one target (ADR 0020)
+JAZZER_FUZZ=1 JAZZER_MAX_DURATION=5m ./mvnw verify -pl integration -am -Dit.test='WriteReadFuzzIntegrationTest#jniReadsJavaWrite' -Dtest=none -Dsurefire.failIfNoSpecifiedTests=false
+./mvnw test -pl fuzz -am -Dtest=SeedCorpusGenerator -Dsurefire.failIfNoSpecifiedTests=false -Dvortex.fuzz.regenerateSeeds=true  # rewrite per-encoding seeds
 ```
 
 Tag-excluded from a routine local build, even when named with `-Dit.test`/`-Dtest`:
@@ -95,9 +96,16 @@ Tag-excluded from a routine local build, even when named with `-Dit.test`/`-Dtes
   integration test taking over ~1 s.
 - **Raincloud corpus** (`@Tag("raincloud")`, failsafe): clear the exclusion with
   `-Dvortex.it.excludedGroups=` as shown above. Not run by the `ci` profile.
-- **Jazzer fuzz** (`@Tag("fuzz")`, own `fuzz` module): clear it with `-Dvortex.fuzz.excludedGroups=`.
-  Without `JAZZER_FUZZ=1` it only replays the saved corpus; with it, it fuzzes until stopped —
-  never in CI or unattended. See [ADR 0020](adr/0020-jazzer-fuzz-infrastructure.md).
+
+Jazzer fuzz targets (`@FuzzTest`, ADR 0020) are **not** excluded: without `JAZZER_FUZZ=1` each
+replays its saved inputs (`*Inputs/<method>/`: seeds plus every past finding) in about a second,
+as part of the routine build. `JAZZER_FUZZ=1` explores new inputs, one target per run, bounded by
+`JAZZER_MAX_DURATION`; the weekly `Fuzz` workflow does that for every target. Targets: reader
+`open`/`scan`/`filter`/metadata-blob and FSST round-trip in `fuzz`; write→read round-trip and the
+vortex-jni differential in `integration` (`WriteReadFuzzIntegrationTest`). A finding is written
+next to the target's inputs as `crash-<hash>`: rename it after the bug, fix, and commit it with
+the fix, plus a `MalformedFileInteropIntegrationTest` case when it is a file vortex-jni also
+reads.
 
 Regenerate after editing `.fbs`/`.proto` (both generators are in-house, no external tools):
 
@@ -110,7 +118,10 @@ Both schema languages are compiled in-process to MemorySegment-native Java, with
 `flatc`/`protoc` and no `com.google.flatbuffers`/`protobuf-java` runtime (ADR 0017):
 - **`.fbs` → `fbs-gen`** (`io.github.dfa1.vortex.fbsgen`): generates readers extending
   `FbsTable` (vtable-based) or `FbsMemorySegment` (fixed-offset inline) and builders over
-  `FbsBuilder`, all in the same generated package `io.github.dfa1.vortex.core.fbs`. The
+  `FbsBuilder`, all in the same generated package `io.github.dfa1.vortex.core.fbs`. Every
+  generated `getRootAsX` first runs `FbsVerifier` over the whole buffer (Rust's
+  `flatbuffers::root`: bounds, vtables, required fields, ≤ 64 nested tables, ≤ 1M tables), so
+  reader code never guards FlatBuffer offsets itself. The
   runtime base classes `FbsTable`/`FbsMemorySegment` are package-private (only generated
   readers extend them); `FbsBuilder` is public because the writer module assembles FlatBuffers
   with it. Schema names with a trailing `_` (e.g. `Struct_`) have the underscore stripped in
@@ -242,7 +253,8 @@ almost always this.
 The reader memory-maps and parses untrusted binary input. Every malformed input must throw
 `VortexException`, never `ArrayIndexOutOfBoundsException`, `NegativeArraySizeException`,
 `OutOfMemoryError`, `StackOverflowError`, a raw FlatBuffer runtime exception, or a Protobuf parser
-exception. See the [`security` issues](https://github.com/dfa1/vortex-java/labels/security) for the current gap list (per-encoding adversarial
+exception. Reject input where it enters, as Rust does: FlatBuffers are verified whole when parsed,
+dtype and layout tree shape at open, and only segment contents at decode. See the [`security` issues](https://github.com/dfa1/vortex-java/labels/security) for the current gap list (per-encoding adversarial
 tests, resource caps, fuzz infra) and [ADR 0003](adr/0003-vortex-exception-sanitization.md) /
 [ADR 0004](adr/0004-resource-caps-read-options.md) for the design.
 

core/src/main/java/io/github/dfa1/vortex/core/fbs/FbsArray.java

@@ -9,11 +9,22 @@
 @Generated("io.github.dfa1.vortex.fbsgen.CodeGen")
 public final class FbsArray extends FbsTable {
 
-    /// Positions a reader at the root table of a finished buffer.
+    /// Verifies a finished buffer and positions a reader at its root table.
     /// @param seg the buffer
     /// @return a reader positioned at the root
+    /// @throws io.github.dfa1.vortex.core.error.VortexException if the buffer is malformed
     public static FbsArray getRootAsFbsArray(MemorySegment seg) {
-        return new FbsArray().assign(seg, FbsTable.rootPosition(seg, 0));
+        return new FbsArray().assign(seg, FbsVerifier.verifyRoot(seg, FbsArray::verify));
+    }
+
+    /// Checks this table and everything reachable from it.
+    /// @param v the verifier walking the buffer
+    /// @param pos the table's absolute position
+    static void verify(FbsVerifier v, long pos) {
+        v.startTable(pos);
+        v.table(pos, 4, "root", false, FbsArrayNode::verify);
+        v.vector(pos, 6, 8, "buffers", false);
+        v.endTable();
     }
 
     /// Positions this reader at a table.
@@ -37,6 +48,12 @@ public int buffersLength() {
         return o != 0 ? vectorLength(o) : 0;
     }
 
+    /// Tells an absent `buffers` vector from an empty one, which [#buffersLength()] cannot.
+    /// @return whether the `buffers` vector is present
+    public boolean hasBuffers() {
+        return fieldOffset(6) != 0;
+    }
+
     /// @param j element index
     /// @return the j-th `buffers` element
     public FbsBuffer buffers(int j) {

core/src/main/java/io/github/dfa1/vortex/core/fbs/FbsArrayNode.java

@@ -9,11 +9,25 @@
 @Generated("io.github.dfa1.vortex.fbsgen.CodeGen")
 public final class FbsArrayNode extends FbsTable {
 
-    /// Positions a reader at the root table of a finished buffer.
+    /// Verifies a finished buffer and positions a reader at its root table.
     /// @param seg the buffer
     /// @return a reader positioned at the root
+    /// @throws io.github.dfa1.vortex.core.error.VortexException if the buffer is malformed
     public static FbsArrayNode getRootAsFbsArrayNode(MemorySegment seg) {
-        return new FbsArrayNode().assign(seg, FbsTable.rootPosition(seg, 0));
+        return new FbsArrayNode().assign(seg, FbsVerifier.verifyRoot(seg, FbsArrayNode::verify));
+    }
+
+    /// Checks this table and everything reachable from it.
+    /// @param v the verifier walking the buffer
+    /// @param pos the table's absolute position
+    static void verify(FbsVerifier v, long pos) {
+        v.startTable(pos);
+        v.inline(pos, 4, 2, "encoding", false);
+        v.vector(pos, 6, 1, "metadata", false);
+        v.tableVector(pos, 8, "children", false, FbsArrayNode::verify);
+        v.vector(pos, 10, 2, "buffers", false);
+        v.table(pos, 12, "stats", false, FbsArrayStats::verify);
+        v.endTable();
     }
 
     /// Positions this reader at a table.
@@ -37,6 +51,12 @@ public int metadataLength() {
         return o != 0 ? vectorLength(o) : 0;
     }
 
+    /// Tells an absent `metadata` vector from an empty one, which [#metadataLength()] cannot.
+    /// @return whether the `metadata` vector is present
+    public boolean hasMetadata() {
+        return fieldOffset(6) != 0;
+    }
+
     /// @param j element index
     /// @return the j-th `metadata` element
     public int metadata(int j) {
@@ -56,6 +76,12 @@ public int childrenLength() {
         return o != 0 ? vectorLength(o) : 0;
     }
 
+    /// Tells an absent `children` vector from an empty one, which [#childrenLength()] cannot.
+    /// @return whether the `children` vector is present
+    public boolean hasChildren() {
+        return fieldOffset(8) != 0;
+    }
+
     /// @param j element index
     /// @return the j-th `children` element
     public FbsArrayNode children(int j) {
@@ -69,6 +95,12 @@ public int buffersLength() {
         return o != 0 ? vectorLength(o) : 0;
     }
 
+    /// Tells an absent `buffers` vector from an empty one, which [#buffersLength()] cannot.
+    /// @return whether the `buffers` vector is present
+    public boolean hasBuffers() {
+        return fieldOffset(10) != 0;
+    }
+
     /// @param j element index
     /// @return the j-th `buffers` element
     public int buffers(int j) {

core/src/main/java/io/github/dfa1/vortex/core/fbs/FbsArraySpec.java

@@ -9,11 +9,21 @@
 @Generated("io.github.dfa1.vortex.fbsgen.CodeGen")
 public final class FbsArraySpec extends FbsTable {
 
-    /// Positions a reader at the root table of a finished buffer.
+    /// Verifies a finished buffer and positions a reader at its root table.
     /// @param seg the buffer
     /// @return a reader positioned at the root
+    /// @throws io.github.dfa1.vortex.core.error.VortexException if the buffer is malformed
     public static FbsArraySpec getRootAsFbsArraySpec(MemorySegment seg) {
-        return new FbsArraySpec().assign(seg, FbsTable.rootPosition(seg, 0));
+        return new FbsArraySpec().assign(seg, FbsVerifier.verifyRoot(seg, FbsArraySpec::verify));
+    }
+
+    /// Checks this table and everything reachable from it.
+    /// @param v the verifier walking the buffer
+    /// @param pos the table's absolute position
+    static void verify(FbsVerifier v, long pos) {
+        v.startTable(pos);
+        v.string(pos, 4, "id", true);
+        v.endTable();
     }
 
     /// Positions this reader at a table.