Repository navigation
Fuzzing for #493: every target, a FlatBuffer verifier, and the bugs they found - #564
Merged
Merged
Conversation
dfa1
changed the title
dfa1
marked this pull request as ready for review
October 18, 2026 16:22
…coding openMalformedFile stopped after open(), so no encoding decoder had seen fuzzed bytes. The fuzz module now has five targets: - scanMalformedFile: columnStats(), then every value of every chunk through ArrayWalker, which reads each leaf by bulk materialize and by the per-row getters (separate code in the Lazy* arrays), recursing through masked, struct, list, map, union and variant arrays, and again limited to half the rows for row-range slicing. - filterMalformedFile: the scan with a filter the fuzzer chooses (column, predicate, a value of the type the column compares against), so zone maps are parsed and chunks pruned. Its input is a 16-byte choice header then the file: a FuzzedDataProvider takes numbers from the end of the input, where the trailer is. - parseMetadataBlobs: the footer, layout and dtype FlatBuffers themselves, wrapped by MetadataBlobs in a valid postscript and trailer, so every mutation lands in the metadata parse; public API only, no access seam into the reader's package. - compressDecompressRoundTrip: an FSST table trained on fuzzed rows must compress every row identically through the byte[] and MemorySegment paths and decompress it back. A correctness oracle: Decompressor is an internal boundary, the reader's FSST validation is the scan target's. A byte mutator never builds a valid FSST symbol table or Pco header from the five hand-made seeds, so SeedCorpusGenerator writes one small file per encoding (FastLanes bitpacked/FOR/delta/RLE, RunEnd, ZigZag, Sparse, Pco, ALP, ALP-RD, Dict, FSST, OnPair, Zstd), each forced through its encoding and checked to really use it, and mirrors every seed into the filter and metadata-blob corpora. Opt-in, regenerated by a system property. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Found by the scanMalformedFile fuzz target: a segment's array FlatBuffer with a child-vector offset past its end escaped as a raw IndexOutOfBoundsException. The FlatBuffer runtime trusts every offset it follows (FbsTable), so each parse site must frame it, as PostscriptParser does for the footer blobs. SerializedArrayDecoder did not, nor did the two other unframed sites: the zone-stats read in ScanIterator (now degrades to no stats, as documented) and the HTTP reader's postscript parse. The catch alone cannot stop a vector length near Integer.MAX_VALUE sizing `new ArrayNode[n]` (an OutOfMemoryError), so a vector must fit in its FlatBuffer; an encoding index past the footer's spec table is rejected too. vortex-jni rejects the same file with its FlatBuffer verifier. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
…ders cast it
Found by the scanMalformedFile fuzz target: a Struct column whose segment
claims fastlanes.bitpacked escaped as a raw ClassCastException, because
the decoder cast its dtype to Primitive unchecked. Delta did the same, and
RLE's empty-chunk shortcut cast before the check the rest of its decode
has. They now reject a non-primitive dtype the way the FOR, ALP and Pco
decoders already do.
RLE also covers bools, so a zero-row bool chunk hit that cast in a valid
file; it now reads as an empty bool array.
vortex-jni rejects the same file ("Expected 1 buffer, got 0").
Refs #493
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
…st does
Found by the scanMalformedFile fuzz target: a struct dtype without a names
vector opened as a 0-field struct, then the scan indexed its field list by
layout child and threw a raw ArrayIndexOutOfBoundsException. Rust refuses
the file at open ("failed to parse struct names from flatbuffer").
Both checks now run at open, where Rust runs them:
- StructFields::from_fb requires the names and dtypes vectors; only an
empty one is a 0-field struct. The generated readers could not tell
absent from empty, so fbs-gen emits hasX() for every vector field.
- Layout::<Struct>::validate_children requires one child per field plus a
leading validity child when the struct is nullable; PostscriptParser
checks it for the root and directly nested struct layouts.
Refs #493
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Rust reads no FlatBuffer without flatbuffers::root::<T>(), which verifies the whole buffer first. vortex-java's FbsTable accessors trust every offset, so each parse site had to frame its own parse in a catch-all; the fuzzer found three sites that did not, each escaping as a raw JDK exception. fbs-gen now emits a package-private verify(FbsVerifier, long) per table, and every getRootAsX runs it before positioning the reader. FbsVerifier holds flatbuffers' rules with Vortex's limits: table, vtable, field, offset, vector and string bounds, NUL terminators, required fields, at most 64 nested tables and 1M tables. Alignment, UTF-8 and the apparent-size cap are not checked yet. With it, the reader drops what only guarded FlatBuffer navigation: the array, layout and dtype depth caps (the verifier's nesting limit, Rust's own, binds first; a dtype level is two tables, so dtypes now nest ~31 deep, as in Rust), the array-parse catch-all, its vector-count guard and the HTTP postscript wrapper. The footer catch-all stays: it also guards DType record validation. Fuzzed 5 min each: openMalformedFile 4.36M inputs, no finding. The inspector's own ArrayNode depth cap goes too, as in the reader; its node-count cap stays, since it bounds aliased children far below the verifier's million-table limit. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
… refuses Found by the scanMalformedFile fuzz target: vortex.zstd_buffers names the encoding it wraps by an id string in its metadata, and one holding a control character escaped EncodingId.parse as a raw IllegalArgumentException. The footer, extension and registry paths that parse ids from a file already convert it; this one now does too. vortex-jni rejects the same file. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Found by the scanMalformedFile fuzz target: a postscript without a dtype
segment opened with a null dtype, and the scan then threw a raw
NullPointerException. Rust opens such a file only when the caller supplies
the dtype (VortexOpenOptions::with_dtype) and otherwise fails at open
("Vortex file doesn't embed a DType ..."). vortex-java has no such option,
so the file is unreadable: parseBlobs, shared by the file and HTTP
readers, now refuses it.
Breaking: such a file used to open. The verifier's CHANGELOG entry is
marked Breaking too: it narrows dtype nesting to Rust's limit.
Refs #493
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
… enters Found by the scan and filter fuzz targets: a dict column whose codes child was empty for a non-empty chunk threw a raw ArithmeticException, because the constant-child broadcast reads `i % capacity`; so did a primitive child with an empty buffer, through the Materialized* arrays' getInt, which broadcast the same way. Eighteen decoder call sites took SegmentBroadcast.capacity(seg, elemBytes) and broadcast with `% capacity`; some guarded an empty child their own way (ALP, FSST, RunEnd, RLE), the others (dict, delta, zigzag, varbin, patched, ALP-RD) did not. And a child neither full nor constant (2 values for 5 rows) was silently wrapped, repeating values, where Rust's children always have their full length. capacity(seg, elemBytes, count) now admits only the two legal shapes, at least `count` elements or exactly one (a constant child), and throws VortexException otherwise; the two-argument form is gone and the per-site guards it subsumes are deleted. The rule is enforced where the data enters, not only where it is read: DecodeContext.decodeChildSegment checks every primitive child segment against the count its decoder declares, broadcastCopy checks its source, and the eight Materialized* leaves take their element count from it in AbstractMaterializedArray, so a malformed buffer fails when the child is decoded or the array built, not when a row is read. Tests that pinned the old wrap-around now expect the error; decoder tests expecting their own later message for an empty child see the shared one; two fixtures were themselves malformed (Patched lane offsets for one chunk on a two-chunk array; FSST zero rows with no offsets, where a varbin always has n + 1). The integration suite, Rust- and Java-written files alike, is unaffected. vortex-jni rejects both files. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Found by the jniReadsJavaWrite differential fuzz target: a sparse column's patch indices, a single u8 index of 230, went through the constant encoding, which read `(byte[]) data)[0]` sign-extended and wrote the u64 scalar 2^64 - 26. vortex-jni refuses the file: "Uint64 value 18446744073709551590 out of range for dtype u8". u16 and u32 constants past their signed range had the same flaw. The constant encoder now takes the element through SparseEncodingEncoder.readBits, which already widens unsigned types correctly, instead of its own copy. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Found by the jniReadsJavaWrite differential fuzz target: a sorted u16
column past 0x7FFF was read sign-extended, and vortex-jni refuses the
u64 scalar 2^64 - 2067 ("out of range for dtype u16").
The encoder now follows encode_primitive_array: values read zero-extended
for unsigned types; the step is slice[1].checked_sub(base) in the
column's own type, so a decreasing unsigned column or an i8 step of 255
is no sequence (both wrote a scalar Rust cannot read); an empty array or
a zero step (constant data, left to the constant encoding) is no
sequence; every later step is a checked_add in that type.
Refs #493
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
… every build; weekly fuzzing Write side: the fuzzer builds a small table, vortex-java writes it, and a reader must return exactly the values written, vortex-java (javaRoundTrip) and vortex-jni (jniReadsJavaWrite). Per column it picks a dtype, nullability, a shape (random, constant, small range, runs, sorted, sparse) and a seed, steering the cascading compressor into its edges while an input stays a few bytes. Both live in integration, where the JNI dependency is; failsafe now loads the byte-buddy agent as surefire does, since Jazzer installs itself through it. The fuzz module excluded its targets from ./mvnw test, guarding against unbounded fuzzing, which only JAZZER_FUZZ=1 starts. Without it a target replays its saved inputs, seeds plus every past finding, in about a second, so the exclusion is gone: a regression of any finding fails the ordinary build. The Fuzz workflow explores new inputs every Saturday (or on demand with a duration), one matrix job per target, uploading any new crash input. fuzz/oss-fuzz holds the project files for an OSS-Fuzz submission (JDK 25 shipped with the fuzzers, one Jazzer --target_method wrapper per target); checked locally without Docker, submission is the maintainer's step. Fuzzed 5 min each: javaRoundTrip 181k inputs, clean; jniReadsJavaWrite found the two writer bugs fixed before this, then 133k inputs clean. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Found reviewing the open-time struct check: a nullable struct layout
carries a leading validity child (Rust's validate_children wants
nfields + 1 children), but the root layout the writer builds has one
child per column and no way to mark a row null. vortex-jni refuses every
such file ("Assertion failed: nchildren == expected"), and since the
open-time check the reader refuses it too.
VortexWriter.create now rejects the schema up front. No importer writes a
nullable root; writing a validity child instead would only ever say "all
valid".
Breaking: such a schema used to be accepted.
Refs #493
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
Found reviewing the child-shape rule: RunEnd's `endsCap <= 0` guard, left after capacity() took over the empty-child case, could only still fire for zero runs, and decode already refuses zero runs for rows, so it only ever rejected an empty array (0 runs, 0 rows) as "empty ends buffer for 0 run(s)". Rust builds and reads empty run-end arrays; an empty array now has nothing to validate. Refs #493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #493. Every item of the issue, one commit each; every bug fix is its own commit with unit tests, the crash input (replayed by its target from now on) and, for files, a
MalformedFileInteropIntegrationTestcase asserting vortex-jni rejects the same file.Fuzz targets (Jazzer
@FuzzTest)openMalformedFileparseMetadataBlobsscanMalformedFilecolumnStats+ every value viamaterializeand per-row getters, whole andlimited(len/2)filterMalformedFileRowFilter(zone tables, pruning)compressDecompressRoundTripbyte[]and segment paths)→decompressjavaRoundTripjniReadsJavaWriteSeedCorpusGenerator: one seed per encoding (checked to really use it), mirrored for the filter and blob targets.fuzzexclusion is gone; ~1 s); a regression of any finding fails CI.Fuzzworkflow (Saturdays, or manual with a duration): one job per target, new crash inputs uploaded.fuzz/oss-fuzz/(build.sh + a generated wrapper verified locally); submission andprimary_contactare the maintainer's step.FlatBuffer verifier (as Rust's
flatbuffers::root)fbs-genemitsverifyper table; everygetRootAsXruns it before any accessor reads the buffer. Removed what it supersedes: three reader depth caps, the inspector's, two catch-alls, a vector-count guard.Bugs found and fixed
Reader (each escaped the security contract as a raw JDK exception):
IndexOutOfBoundsExceptionClassCastException(an empty RLE bool chunk crashed in a valid file)zstd_buffersinner id with a control character →IllegalArgumentException/ by zero(decoders' broadcast and theMaterialized*arrays): one shape rule now (full, or one constant value), inSegmentBroadcast.capacity, which also stops silently wrapping truncated childrenWriter (vortex-jni could not read the file):
u8/u16/u32past the signed range written sign-extendedvortex.sequence: sign-extended unsigned values, and steps Rust's checked arithmetic refuses; now mirrorsencode_primitive_arrayBreaking (marked in CHANGELOG, so the release is 0.18.0)
Verified
Full
./mvnw verifygreen on every module: 9,024 tests, 0 failures, javadoc and checkstyle included.🤖 Generated with Claude Code
https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K