Skip to content

structdiff: compare scalars and string maps without reflect allocations - #7205

Merged
denik merged 1 commit into
mainfrom
denik/struct-diff-json
Oct 28, 2026
Merged

denik merged 1 commit into
mainfrom
denik/struct-diff-json

Conversation

@denik

@denik denik commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Cuts allocations in structdiff.IsEqual (to zero for equal inputs) and GetStructDiff, with no change in output.

  • String/bool/int/uint/float leaves are compared directly instead of boxing both sides for reflect.DeepEqual.
  • map[string]string (tags, spark_conf, ...) is compared natively; reflect copied every key and value to the heap.

JobSettings fixtures (go test -bench . -benchmem ./libs/structs/structdiff), median of 8:

before after
IsEqual, equal 52.8µs, 198 allocs 47.0µs, 0 allocs
GetStructDiff, equal 49.0µs, 560 allocs 43.4µs, 356 allocs
GetStructDiff, changes 49.3µs, 585 allocs 44.0µs, 381 allocs
GetStructDiff, zero 50.2µs, 638 allocs 46.5µs, 485 allocs
GetStructDiff, nils 37.1µs, 398 allocs 35.4µs, 359 allocs

Related: #7206, #7207

This pull request and its description were written by Isaac.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 87b67a4

Run: 40674626841

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 15 5:42
✅​ aws-windows-latest-4core-16gb 283 13 3:37
✅​ azure linux-2core-8gb 280 15 5:42
✅​ azure-windows-latest-4core-16gb 282 13 3:36
✅​ gcp linux-2core-8gb 281 15 5:45
✅​ gcp-windows-latest-4core-16gb 283 13 3:44
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:55 azure linux-2core-8gb TestAccept
3:47 aws linux-2core-8gb TestAccept
3:46 gcp linux-2core-8gb TestAccept
3:19 gcp-windows-latest-4core-16gb TestAccept
3:11 aws-windows-latest-4core-16gb TestAccept
2:55 azure-windows-latest-4core-16gb TestAccept

@denik denik changed the title structdiff: cache struct field info and build paths lazily structdiff: build paths lazily and stop allocating on equal values Oct 28, 2026
@denik denik changed the title structdiff: build paths lazily and stop allocating on equal values structdiff: compare scalars and string maps without reflect allocations Oct 28, 2026
@denik
denik marked this pull request as ready for review October 28, 2026 10:43
@denik
denik requested a review from a team as a code owner October 28, 2026 10:43
@denik
denik requested a review from pietern October 28, 2026 10:43
Leaves of basic kinds are compared directly instead of boxing both sides for
reflect.DeepEqual, and map[string]string is compared natively; reflect copied
every key and value to the heap. IsEqual on equal JobSettings goes from 198
allocations to 0, GetStructDiff from 560 to 356.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik force-pushed the denik/struct-diff-json branch from c38b223 to 87b67a4 Compare October 28, 2026 10:44
@denik
denik added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 35ed095 Oct 28, 2026
27 checks passed
@denik
denik deleted the denik/struct-diff-json branch October 28, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants