Repository navigation
fix(billing): bill the factorization numpy runs for method= and hermitian=True - #302
Open
AndreasHad04 wants to merge 1 commit into
Open
AndreasHad04 wants to merge 1 commit into
AndreasHad04 wants to merge 1 commit into
Conversation
…tian=True (AIcrowd#280) Generator.multivariate_normal(..., method=) factors the covariance with an SVD, eigh or Cholesky, and linalg.svd, pinv and matrix_rank with hermitian=True run eigh or eigvalsh instead of an SVD. All of them were billed as the full SVD: for a 256x256 covariance, method='cholesky' cost 872,685,568 while the factorization it runs bills 11,184,810 standalone, and svd(C, hermitian=True) cost the same as the general SVD. The multivariate_normal factorization term now follows method (26*d^3, 9*d^3 or d^3/3, what svd, eigh and cholesky bill standalone), with the same sampling term; RandomState and the module-level function, which always use SVD, are unchanged. For hermitian=True on a square matrix the new hermitian_svd_cost bills the branch numpy runs: eigh (eigvalsh for values only) plus ordering the singular values by magnitude, each step at its op's own weight. svd uses it, keeping the truncated cost of a top-k request when that is lower, and pinv and matrix_rank use it for their SVD term. hermitian=False and a non-square matrix, which numpy refuses, bill as before.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #280.
What was wrong
Two options make numpy run a cheaper factorization than an SVD, and both were billed as the full SVD.
Generator.multivariate_normal(..., method=)factors the covariance with an SVD,eighor Cholesky, butmultivariate_normal_flopsalways chargedsvd_cost(d, d, with_vectors=True): for the issue's 256x256 covariance every method cost 872,685,568, while the Cholesky factorization numpy actually runs bills 11,184,810 standalone.linalg.svd,linalg.pinvandlinalg.matrix_rankwithhermitian=Trueonly forwarded the flag; numpy then runseigh(oreigvalshwithout vectors) instead of an SVD, yet the bill was identical tohermitian=False(872,415,232 forsvdagainst 301,989,888 for theeighit runs).What it changes
multivariate_normal_flopstakes the method, and its factorization term is what that factorization bills standalone:svd_cost(d, d, with_vectors=True)=26d^3for'svd'(the default),eigh_cost(d)=9d^3for'eigh'andcholesky_cost(d)=d^3/3for'cholesky'; the sampling term is unchanged. TheGeneratorformula reads the keyword-onlymethod=;RandomStateand the module-level function, which always use SVD, bill as before. The issue's three methods now bill 872,685,568, 302,260,224 and 11,455,146.For
hermitian=True, the newhermitian_svd_cost(n, with_vectors=)in_decompositions.pybills the branch numpy runs:eigh(oreigvalsh) plus ordering the singular values by magnitude (sign,abs, anargsortand threetake_along_axisgathers at weight 4, the sign folded into the vectors and the conjugate copy that formsVh; values only,absand asort), each step at its op's own weight the way composite ops fold sub-tiers.svdbills it for a square matrix (302,797,824 in the issue, just above the 301,989,888 ofeighalone), keeping the truncated cost of a top-k request when that is lower, andpinvandmatrix_rankuse it for their SVD term (369,907,200 and 44,756,650 in the issue).hermitian=False, and non-square input, which numpy refuses underhermitian=True, bill as before. The registry notes forlinalg.svd,linalg.pinvandGenerator.multivariate_normal,ops.json(regenerated, threenotesfields) and the cost-model rows describe the new terms.Tests
New
tests/test_mvn_method_hermitian_billing.py(22 tests): at production weights the issue's three methods bill exactly the standalone factorization plus the shared sampling term; under unit weights each method matches its closed form for twod/sizecombinations; draws equal numpy's for every method;RandomStateand module-level sampling are unchanged. Forhermitian=True,svdwith and without vectors bills exactly the same as numpy's hermitian branch spelled out op by op with fnp, forn= 1, 2, 5, 64 and a batch, and returns numpy's values; the issue's rows land just above theeighbill and keeppinv's andmatrix_rank's other terms; closed forms match forsvd,pinvandmatrix_rank, with numpy's results; top-k requests bill the lower of the truncated cost and the hermitian path;hermitian=Falseis unchanged and non-square input still raisesLinAlgError. 14 of the 22 fail without the source change.Results
uv run pytest): 10387 passed, 125 skipped, 2 xfailed.tests/parityfast gate: 136 passed.ruff check,ruff format --check,pyrighton the changed files: clean.Note: this branch and #291 both change
matrix_rankin_properties.pyand its cost-model row, so whichever merges second needs a small rebase.