Skip to content

BUG: raise MemoryError on argpartition allocation failures - #615

Closed
00200200 wants to merge 2 commits into
pydata:masterfrom
00200200:codex/argpartition-allocation-errors
Closed

00200200 wants to merge 2 commits into
pydata:masterfrom
00200200:codex/argpartition-allocation-errors

Conversation

@00200200

@00200200 00200200 commented Oct 11, 2026 •

Copy link
Copy Markdown

Fixes #529.

argpartition dereferences an unchecked output allocation and an unchecked temporary buffer allocation. A forced failure of the output allocation currently terminates the interpreter with SIGSEGV. A failed buffer allocation can likewise reach the partition loop with a null pointer while the GIL is released.

Check the output before initializing the iterator. Allocate the scratch buffer with PyMem_Malloc while holding the GIL, check it and release the output on failure, and call PyMem_Free after restoring the GIL. Allocation failures now raise MemoryError.

Add subprocess regressions for both allocation sites and all four accelerated dtypes. Each case repeats the error path, checks input and output-dtype reference counts, and checks an ordinary call after removing the allocation hooks.

Validation:

  • Before: 8 regression failures (4 subprocesses terminated with SIGSEGV; 4 showed the scratch buffer bypassing Python allocation hooks).
  • After: all 8 regressions passed on macOS / CPython 3.11.5 / NumPy 2.4.6, and on Linux amd64 / CPython 3.12.15 in a local Docker container.
  • Focused nonreducing-axis and memory tests: 36 passed, 1 expected xfail.
  • Complete suite on macOS: 216 passed, 1 expected xfail.
  • Repository-pinned Ruff 0.16.10 lint and format checks passed for the whole repository. All applicable pre-commit hooks and git diff --check passed.

The fault-injection tests require CPython's _testcapi and skip if that module is unavailable. Windows, free-threaded Python, sanitizers and the full Linux suite were not tested locally. No generated C files are included.

This contribution was prepared and tested by OpenAI Codex, an AI coding agent, at the account owner's request. No personal human code review is claimed.

CI update: merged current master in b3bf46d to pick up the upstream zizmor fix. All 35 checks now pass, including pre-commit.ci; the tag-only release job is skipped. All applicable pre-commit hooks also pass locally. The refreshed Python 3.12 suite reports 208 passed, 8 skipped and 1 expected xfail; all eight allocation-failure regressions pass separately on Python 3.11. The allocation-failure fix is unchanged.

@00200200 00200200 closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUFFER_NEW macro: unchecked malloc inside GIL-released region

1 participant