Skip to content

fix(store): dedup LIMIT_DEDUP index elements by id - #3451

Open
ShryukGrandhi wants to merge 1 commit into
apache:masterfrom
ShryukGrandhi:fix/issue-3443-store-limit-dedup-on-no-scan-with-loadpr
Open

ShryukGrandhi wants to merge 1 commit into
apache:masterfrom
ShryukGrandhi:fix/issue-3443-store-limit-dedup-on-no-scan-with-loadpr

Conversation

@ShryukGrandhi

Copy link
Copy Markdown
Contributor

Purpose of the PR

In BusinessHandlerImpl.scanIndex(...), the transElement branch wraps scanIndexToBaseElement in MapLimitIterator for LIMIT_DEDUP. MapLimitIterator dedups on the elements themselves, but BaseVertex and BaseElement don't define equals/hashCode, so two vertex objects with the same id both get through. DEDUP and PRECISE_DEDUP on the same branch already dedup by id.

The path isn't reachable yet (index2Element throws and loadPropertyFromIndex is always false), but once it is, any index that writes several entries per element (search, collection secondary, joint secondary) would return the same vertex more than once.

Main Changes

  • MapLimitIterator gets a second constructor that takes a key extractor; the existing constructor uses Function.identity(), so the BackendColumn callers behave as before.
  • The LIMIT_DEDUP case on the transElement path passes BaseElement::id.
  • BaseVertex equality is left alone, since changing it would affect every collection that holds vertices.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • New MapLimitIteratorTest (added to CoreSuiteTest) feeds two fresh BaseVertex objects per id through MapLimitIterator. Before the change it emitted 20 elements for 10 ids; now it emits 10.
    • FilterIteratorColumnEqualityTest, FilterIteratorContractTest, ProjectionStageTest, UnionFilterIteratorTest and FilterIteratorCleanupTest still pass.

Does this PR potentially affect the following parts?

  • Dependencies
  • Modify configurations
  • The public API
  • Other affects (typed here)
  • Nope

Documentation Status

  • Doc - TODO: required documentation is pending; complete it before merging.
  • Doc - Done: documentation is included here or linked below.
  • Doc - No Need: no user-visible documentation is affected.

Signed-off-by: ShryukGrandhi <85529990+ShryukGrandhi@users.noreply.github.com>
@codecov

codecov Bot commented Nov 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 58.42%. Comparing base (8e327b8) to head (c76f6c6).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
.../hugegraph/store/business/BusinessHandlerImpl.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             master    #3451   +/-   ##
=========================================
  Coverage     58.42%   58.42%           
- Complexity     6903     6905    +2     
=========================================
  Files           779      779           
  Lines         69391    69391           
  Branches       9567     9567           
=========================================
  Hits          40539    40539           
- Misses        25179    25181    +2     
+ Partials       3673     3671    -2     
Flag Coverage Δ *Carryforward flag
commons 96.37% <ø> (ø) Carriedforward from 8e327b8
hstore 36.08% <ø> (ø) Carriedforward from 8e327b8
pd 56.74% <ø> (ø) Carriedforward from 8e327b8
server_hbase 42.58% <ø> (ø) Carriedforward from 8e327b8
server_memory 56.52% <ø> (ø) Carriedforward from 8e327b8
server_rocksdb 43.03% <ø> (ø) Carriedforward from 8e327b8
store 35.96% <0.00%> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Store] LIMIT_DEDUP on NO_SCAN with loadPropertyFromIndex compares BaseVertex by reference

1 participant