Skip to content

Recompute move_sum after a large value leaves - #629

Open
CAOShurong wants to merge 1 commit into
pydata:masterfrom
CAOShurong:fix-move-sum-large-window
Open

CAOShurong wants to merge 1 commit into
pydata:masterfrom
CAOShurong:fix-move-sum-large-window

Conversation

@CAOShurong

Copy link
Copy Markdown

move_sum keeps a running total and subtracts the value that leaves the window. With a value near 1e19 in the window, that subtraction is coarser than the numbers that remain, so a later window of positive values came back as 0. When the outgoing value dominates the updated total, the current window is summed directly.

pytest --pyargs bottleneck on Windows with Python 3.13, run from doc so the installed package is imported: 209 passed, 1 xfailed. The xfailed test is test_memory_leak. Before this change the new regression reported 0.0 instead of 12.0. A local pytest plugin was disabled because it imports requests. Integer move_sum still uses the running float64 total. Sums that are themselves near the top of the float range are unchanged.

Fixes #480

The running sum subtracts the outgoing value in the input dtype. After a value near 1e19 leaves, the remaining positive window can come back as 0. That window is added directly when the outgoing value dominates the result.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] move_sum returns a negative value for a sum over a pandas Series with all non-negative values

1 participant