[tae898]

Closes #231.

Native insert_many now keeps rows with lists of ints or of floats (the token and weight columns of a sparse-vector load, fixed or ragged) on the columnar entry arcadedb_n_insert_colp, instead of sending the whole chunk as one JSON text. Before, columnar._list_column returned None for a list of ints, so pack refused the chunk and _insert_chunk fell through to arcadedb_n_insert_rows.

New buffer format (four kinds next to the existing q d b i e f s): the data buffer is int32 offsets[n+1] counted in elements (the same shape as a string column's offsets; offsets[0] need not be 0; a null or empty row has equal neighbours) and the aux buffer holds the elements, native-endian. I is int32 elements stored as int[] (property declared ARRAY_OF_INTEGERS), E is float32 stored as float[] (declared ARRAY_OF_FLOATS), Q is int64 stored as a list of Long and D is float64 stored as a list of Double (undeclared or declared LIST), so the engine stores what the JSON text parses to today. Optional Arrow null bitmaps work as for the other kinds. The Java side checks every offset against the element bytes before the first document is built, so a wrong table is an exception and nothing is written.

A library without the decoder is detected by the missing probe entry arcadedb_n_colp_lists (lib.COLP_LISTS, also ARCADEDB_NATIVE_COLP_LISTS=0), and keeps such columns on the JSON path. Anything the kinds cannot carry unchanged still goes the JSON way: mixed int and float elements, bools, nested lists, tuples, NaN or an infinity, an int past 64 bits, an int outside int32 for ARRAY_OF_INTEGERS, ints in an ARRAY_OF_FLOATS ragged list, another declared array type. A fixed-length float list for ARRAY_OF_FLOATS keeps its float32 matrix.

Files: native/.../BulkOps.java (decoder and validation), ArcadeNative.java (probe entry), src/arcadedb/_native/columnar.py (_ragged_column), bulk.py, lib.py.

Tests (run in CI, no local runs): tests/test_native_columnar_pack.py covers the layouts and every refusal; tests/test_native_columnar_insert.py loads the sparse shapes (fixed and ragged, with an LSM_SPARSE_VECTOR index), every list shape (ARRAY_OF_INTEGERS, ARRAY_OF_FLOATS, declared LIST, undeclared, None rows, empty rows, int32 and float32 edges), more rows than one chunk, a library without the decoder, and malformed tables. A spy proves the columnar entry ran and no JSON text was sent (arcadedb_n_insert_rows and arcadedb_n_exec_many counts are 0), and the stored records are compared with the JSON path value by value, Python type by Python type and by the engine's type(), plus the vector.sparseNeighbors results of the two indexes.

Draft until CI is green and the timed wheel-vs-wheel A/B (pre-registered under DECISIONS #240 in the maintainer notes) has a verdict. Untimed orientation figures are in the issue.

🤖 Generated with Claude Code

[tae898]

First timed run (native wheel against native wheel: arm A = main's CI native wheel, arm B = this PR's CI native wheel, A/A control; laptop timed lane, tools/ab.py --noise, ABBA, round 0 dropped). B/A median with 95% CI (below 1 = this PR is faster or smaller):

figure	role	B/A	A/A
sparse rows (30 nonzeros), load	gating	0.572 [0.562, 0.576]	0.998 [0.986, 1.003]
ragged int and float lists, load	gating	0.598 [0.591, 0.607]	1.004 [0.995, 1.007]
plain columns, load	control	0.919 [0.894, 0.955]	0.990 [0.987, 1.006]
peak RSS	reported	1.048 [1.031, 1.088]	0.977 [0.952, 1.028]

Clean pairs per figure: 11; every run passed its arm check (each arm ran its own library and crossed its own entries).

By the rule written before the run this is GRAY: both sparse loads are well below the 0.90 bar, but the plain-column control is outside [0.97, 1.03]. The two arms are different builds of the native library, and plain columns cross the same entry on both, so a moving control says the new library as a whole changed speed, not only the list path; the rule then makes no speed claim. Peak RSS is reported, not gating, and it is higher on this PR.

The study declared one confirmatory rerun for a GRAY before the first run (same arms, pins and rule, final). It is queued; both runs will be reported here, and the PR stays draft until it reads.