Repository navigation
Conversation
Filter empty associations before constructing typed slices while preserving the original grouping policy and windows. Keep IncludeEmpty as the default and document consumer and cache preconditions. Add Arrow-backed integration coverage for Pair and Triple policies, iterator copies, cache refresh, filtered grouping, bindings and window metadata.
Remove the explanatory comments added with the SkipEmpty policy and its integration coverage. Keep existing comments, license headers and code unchanged.
Use the canonical ALICE O2 copyright wording in the integration test. Align the empty eligibility structs with clang-format 20.
|
This improves a lot for the event-mixing when we handle the very rare daughter case, such as |
|
@aalkin Could you take a look this? |
|
I commend the speed up, but the approach seems unnecessarily convoluted. Do you have profiling graph for the current code? Where exactly the time is spent? What exactly the check against joined or concatenated table achieves? Also in the Last, but not least, there is an upcoming change in how the slicing cache is populated in #15867 which will most likely break your new tests. If you can make the tests work with both current and upcoming - perfect, if not, you can at least prepare the version for later. |
GroupedCombinationscurrently constructs typed associated slices before the consumer can check whether they are empty. For sparse associations, repeatedly constructing these slices can dominate the mixing workload, even when the consumer immediately discards empty tuples.This change adds an explicit
GroupedSlicePolicy::SkipEmptyoption. It checks association counts through the existing sortedSliceCachebefore constructing typed slices.IncludeEmptyremains the default, preserving the existing yielded stream.Consumers should enable this option only when processing an empty-associated tuple has no observable effects, including normalization, counters or window bookkeeping.
Performance
A three-run evaluation of an
Omega2012mixed-event workload on macOS arm64, using a fixed seven-DF input and mixing depth 10, gave the following mean CPU times:Xi, K0sXi1530, KThese are complete local DPL CPU times (
user + system, including child processes). All valid measurements were included. TheXi, K0sbaseline had a coefficient of variation of 5.19%.The input has sparse
cascadeassociations, so these gains are workload-dependent. Dense inputs may benefit less or incur additional cache-lookup overhead.Validation