Repository navigation
Conversation
#222) create_vertices rows and new_edge/new_edges props that hold a one-dimensional float32 ndarray went to arcadedb_n_gb as F-tagged JSON text: Python formatted every float and the library parsed it back (text 1.7x to 5.4x the float32 bytes at the lanes' shapes, #222). arcadedb_n_gb_bin(handle, op, json, floats, n_floats) is arcadedb_n_gb plus the float32 side buffer that arcadedb_n_exec_bin already reads: the JSON carries ["B", offset, count] and Json.decodeTag returns the same float[] the F tag gave, so stored values are bit-identical. NativeGraphBatch._op uses it when the call has such an array and the library has the entry (lib.GB_BIN, ARCADEDB_NATIVE_GB_BIN=0 turns it off); a library built before it, lists, float64 arrays, empty arrays and rows without a vector keep the text path. No change to the JVM wheel. Tests: tests/test_native_graph_batch_binary_vectors.py (native only: spy on lib.call for the entry name, bit-equal storage at 64/96/128-d, offsets of mixed rows and of every chunk, edge props, fallback by the probe flag, an out-of-buffer tag refused) and a shared bit-equality test in test_float_vector_bulk.py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Timed A/B (native build; arm A = main 7e64c5e, arm B = PR 224 9ac30ec, both the CI native wheels of their commits; A/A control; laptop timed lane,
Result by the rule: SHIP (every gating interval below 1.00, the control inside [0.97, 1.03], the A/A control clean). Sending float32 vectors in the binary side buffer instead of as F-tagged JSON text removes most of the load time for vector-carrying vertices; loads without vectors stay inside the control band (the control interval is just above 1.00). Marking this ready. It merges after the other pending timed A/Bs that pin Edit: the sentence on loads without vectors said "unchanged"; the control interval lies just above 1.00, inside the rule's band, so it now says that. |
|
Edited the comment above: it said loads without vectors are "unchanged"; the control interval lies just above 1.00 (inside the rule's band), and the sentence now says so. The verdict is the same. |
…ary buffer on native (PR 224) Vectors how-to gets the rule and the study's wall-time ratios with their intervals (native-gb-binary-vectors-ab-laptop-20261104/ab/ab-score.md); the native create_vertices docstring states the float32 rule. Docs and docstring only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…packages The JVM build already sends an ARRAY_OF_FLOATS property of create_vertices as one binary float32 matrix (graph_batch.py), so the advice to pass float32 arrays holds for arcadedb-jvm too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #222
NativeGraphBatch._opsent every op argument as JSON text, so a float32 ndarray increate_verticesrows or innew_edgeandnew_edgesprops went through.tolist()into an F-tagged number list. This addsarcadedb_n_gb_bin(handle, op, json, floats, n_floats),arcadedb_n_gbplus the float32 side buffer thatarcadedb_n_exec_binalready reads. The JSON carries aBtag with an offset and a count, andJson.decodeTagreturns thefloat[]the F tag gave, so the stored values are bit-identical.What changes:
native/.../NativeExtras.java:arcadedb_n_gbandarcadedb_n_gb_binsharegbImpl, which passes the buffer to the parser.Json.object(the streaming parser ofcreate_vertices) takes the buffer too; its old signature stays._native/codec.py:encode_gb_argswrites one-dimensional, non-empty float32 ndarrays to the side buffer and everything else as before. Lists, float64 arrays and rows without a vector keep their text._native/lib.py:GB_BIN(the loader clears it when the library has noarcadedb_n_gb_bin;ARCADEDB_NATIVE_GB_BIN=0turns it off for an A/B in one image), and the ctypes signature, next toarcadedb_n_exec_bin._native/extras.py:_oppicks the entry. A library built before the entry keeps the text path.Tests (run in the native job, against the wheel this branch builds):
tests/test_native_graph_batch_binary_vectors.py(native only): a spy onlib.callseesarcadedb_n_gb_binfor float32 rows of 64, 96 and 128 dimensions and the stored vectors are bit-equal to the inputs (-0.0, the float32 maximum, the smallest normal and the smallest subnormal included). It also covers the binary path against the text path in the same image, the fallback by the probe flag, None, list, float64 and empty vectors staying on the text entry, mixed rows (every offset), the chunks of a 2,500-row load (one buffer each), edge props, and a tag outside the buffer being refused by the library.tests/test_float_vector_bulk.py::TestCreateVertices::test_float32_rows_are_stored_bit_equal: the same bit-equality at the three dimensions on both packages.What I could and could not run: the laptop has no native build, so the Java was compiled with
javac(Corretto 25, Maven artifacts of 26.10.2) and the Python tests were run against the CI wheel of main with this branch's three Python files laid over it, which is a library without the new entry. There the fallback tests pass, the 7 tests that need the entry fail as they must, and the shared bit-equality test passes. Against main's own Python code, 12 of the 13 new native tests fail (they needlib.GB_BINor the new call name) and the one that pins the unchanged text path passes. The binary entry itself is exercised only by the native job of this branch, which is dispatched.Draft until that job is green. A wheel-versus-wheel A/B (timed) follows from the CI wheels of main and of this branch; the numbers in the issue are an untimed probe.
🤖 Generated with Claude Code
Docs updated in 8e04007 (docs and docstrings only).
Edited 2026-11-04: added the docs line above.