Skip to content

Fix unreachable ordered rate accumulators - #19250

Merged
JackieTien97 merged 3 commits into
apache:masterfrom
CoollZzz:fix/ordered-reachable
Nov 13, 2026
Merged

JackieTien97 merged 3 commits into
apache:masterfrom
CoollZzz:fix/ordered-reachable

Conversation

@CoollZzz

@CoollZzz CoollZzz commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

Problem

For typical rate aggregation plans whose child ordering does not include the rate function's time argument, inputOrderedByTimeAscending remained false. These plans therefore used the naive accumulator, which buffers each group's (time, value) samples. The existing ordering-based path could still select an ordered accumulator when the child ordering proved the required order.

Change

Extend the existing ordering-based check with a scan-based ordering guarantee for a single-region table scan.

For a SINGLE aggregation over an ascending plain table scan, the scan input is considered time ordered per group when only order-preserving Project or Filter nodes lie between the aggregation and scan, and either:

  1. the scan contains one device; or
  2. the grouping keys cover every TAG column, so each group belongs to at most one device.

The rate function's second argument must also resolve through projections to the scan's native time column. Rate functions may use another INT64 or TIMESTAMP column as that argument, whose values are not necessarily ordered with the scan.

This supplements the existing ordering-based check, preserving its behavior when the plan itself proves the required ordering.

Verification

  • RateOrderedInputTest: checks single-device input, all-TAG and subset grouping, descending scans, non-SINGLE steps, projection resolution, and rejection of unsupported child plans.
  • RateOrderedPlannerTest: checks the planner flag for single-device, multi-device, and non-time-argument queries.
  • Runtime check on a 1C1D instance: SELECT irate(s1, time) FROM t WHERE device_id = 'd1' constructs OrderedIrateAccumulator; the multi-device query constructs NaiveIrateAccumulator.

@CoollZzz CoollZzz changed the title Enable ordered rate accumulators for single-device and tag-grouped scans Fix unreachable ordered rate accumulators Nov 13, 2026

@JackieTien97 JackieTien97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. I reviewed the planner change and verified it locally, and I didn't find a correctness problem: the per-group ordering argument (a single device, or grouping keys that cover every TAG column of a single-region DeviceTableScanNode) holds, and the remaining cases fall back to the naive accumulators.

What I checked:

  • Real distributed plans built with TestMetadata: a single device, and two devices in one region grouped by all TAG columns, are marked ordered; GROUP BY tag1, a global aggregation over two devices, DESC scans, and plans split into PARTIAL/FINAL are not.
  • A local 1C1D run with one DataRegion and four devices, with data in sequence files, unsequence files, the memtable, and two time partitions. Every rate-family query returned identical results with the native time column (ordered path) and with an equal INT64 column (naive path). jcmd VM.classes showed that only the Ordered* accumulators ran for the multi-device query grouped by all TAG columns.

The inline comments are mostly about tests: the added tests don't actually protect the new planner wiring. There is also a time-argument-index conflict with #19227.

CI has not run on this PR yet.

@JackieTien97 JackieTien97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update. I re-checked f86deb9, and all six points from my previous review are addressed:

  • The planner tests now build a SINGLE aggregation over a two-device scan, and planSingleAggregation only accepts SINGLE nodes. The mutant that treats every SINGLE aggregation as time ordered, which passed all tests before, now fails testTwoDevicesGroupedBySubsetOfTagsKeepNaiveInput and testGlobalAggregationOverTwoDevicesKeepsNaiveInput.
  • getOrderedInputTimeArgumentIndex is shared by both checks, so whichever of this PR and #19227 lands second only needs to register INTEGRAL (index 2) there.
  • The four-argument functions, the time alias, the wrapped Sort/Limit/Collect children, and the excluded scan types are covered now, and TagColumnUtils replaces the three copies. The ParallelizeGrouping and TransformSortToStreamSort changes look behavior-preserving to me.

Locally, 131 related tests pass: the Rate* tests plus ParallelizeGroupingTest, OrderingPropertyPropagationTest, TableFunctionSortTest, SortTest, SubQueryTest, LimitOffsetPushDownTest, and InsertIntoQueryTest. spotless:check also passes on the six changed files.

One small test issue is inline. A few other notes:

  • Could you update the PR description? It still describes the first version of the tests and doesn't mention that ParallelizeGrouping and TransformSortToStreamSort now use TagColumnUtils.
  • An IT with several devices in one region grouped by all TAG columns would still be nice to have, but I don't consider it blocking.
  • CI hasn't run on this PR yet.

@JackieTien97
JackieTien97 merged commit 59dbe07 into apache:master Nov 13, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants