Repository navigation
IP-2b: small correctness fixes (F12, F23, D3, 585#18) - #673
benjaminpope wants to merge 5 commits into
Conversation
…ndependent MultipoleField taper, projected OIData validation Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Size of this PR (Python lines)
|
|
Claude encountered an error after 3m 26s —— View job Review of PR #673I'm asking for changes. The What to fix
Checked, no problems found
|
| if not ( | ||
| onp.all(onp.isfinite(values)) | ||
| and onp.all(onp.isfinite(errors)) | ||
| and onp.all(onp.asarray(errors) >= 0) |
There was a problem hiding this comment.
>= 0 lets a zero error through, and nothing later rejects it for already-projected data. _transform_observed_channels skips data whose size differs from n_vis/n_phi (oidata.py:984, :1005), so _diagonalized and its _ZERO_PROJECTED_ERROR check never run on this path. The new docstring says "a zero projected error is rejected where the data are whitened", but I couldn't find any such check for this case.
Failure: OIData({..., "vis": [0.9]*5, "d_vis": [0.01, 0.0, 0.01, 0.01, 0.01], "vis_mat": A}) builds, and then whitened_residuals divides by zero, giving inf/NaN χ² and a NaN gradient. Projected values are already the whitened basis, so a zero error is no more usable here than on the non-operator path. That path uses _valid_samples (> 0).
| and onp.all(onp.asarray(errors) >= 0) | |
| if not onp.all(_valid_samples(values, errors)): |
(and change the message and docstring to say "positive"). tests/test_oidata_projected.py has no zero-error case, so add onp.array([0.01, 0.0, 0.01, 0.01, 0.01]) to the parametrize list.
There was a problem hiding this comment.
Switched to _valid_samples (positive finite errors), message and docstring say positive, zero-error case added to the parametrization. 4aa810a
🤖 Addressed by Claude Code
| ## The multipole field prior | ||
|
|
||
| A `MultipoleField` is the same Gaussian-process idea on a polar grid in the disk's frame, here inclined 50° at position angle 30°. It is a residual that follows the disk's geometry. The top row shows three prior draws. The bottom shows the field's azimuthal Fourier coefficients: the cosine amplitude $a_m(r)$ of each harmonic $m = 0, \dots, 3$ in $\eta = \sum_m a_m \cos m\theta + b_m \sin m\theta$, against disk-plane radius, each on its own offset track (the $m = 0$ coefficient is the azimuthal mean). Every harmonic is a smooth function of radius with correlation length $\ell_r$. At the centre the $m \geq 1$ harmonics are tapered to zero, as $\tanh^m(r/\Delta r)$ with $\Delta r$ the radial node spacing, so that the field has a single value at the origin; this is why those tracks pinch to zero at $r = 0$. Higher harmonics have smaller amplitudes, with a scale set by $\ell_\theta$. | ||
| A `MultipoleField` is the same Gaussian-process idea on a polar grid in the disk's frame, here inclined 50° at position angle 30°. It is a residual that follows the disk's geometry. The top row shows three prior draws. The bottom shows the field's azimuthal Fourier coefficients: the cosine amplitude $a_m(r)$ of each harmonic $m = 0, \dots, 3$ in $\eta = \sum_m a_m \cos m\theta + b_m \sin m\theta$, against disk-plane radius, each on its own offset track (the $m = 0$ coefficient is the azimuthal mean). Every harmonic is a smooth function of radius with correlation length $\ell_r$. At the centre the $m \geq 1$ harmonics are tapered to zero, as $\tanh^m(r/\ell_r)$, so that the field has a single value at the origin; this is why those tracks pinch to zero at $r = 0$. Higher harmonics have smaller amplitudes, with a scale set by $\ell_\theta$. |
There was a problem hiding this comment.
This page is generated from notebooks/multipole_disk_rings.ipynb (scripts/sync_tutorial_docs.py MAPPINGS, line 64), so AGENTS.md says not to edit it directly. The notebook's markdown cell (around line 603 of the .ipynb) still says \tanh^m(r/\Delta r) with tests/lint/test_tutorial_docs_sync.py will fail.
The bigger problem: the figure under this paragraph is now wrong. The notebook uses length_r=6.0 on 32 nodes over 40 mas (Δr ≈ 1.29 mas), so the taper scale goes from 1.29 to 6 mas, about 4.6× wider. The "pinch to zero at r = 0" in the a_m tracks and the prior draws were rendered with the old taper. Edit the notebook cell, re-execute it (an OzSTAR run plus the awaiting-ozstar label if it is heavy), and then run the sync script.
There was a problem hiding this comment.
Edited the notebook cell and regenerated the page (sync test passes). The figures were rendered with the old taper and need a re-execution of the notebook (heavy; left for an OzSTAR run, flagged in the PR body). 4aa810a
🤖 Addressed by Claude Code
| j = np.arange(n_theta) | ||
| m = np.minimum(j, n_theta - j) | ||
| taper = np.tanh(nodes / self.spacing)[:, None] ** m[None, :] | ||
| taper = np.tanh(nodes / self.length_r)[:, None] ** m[None, :] |
There was a problem hiding this comment.
This change leaves docs/api/models/notes.md:393-401 (MultipoleField, "Regularity at the centre") stale. That section still says length_r is 5 spacings, the default prior now becomes that close to stationary only beyond about 25 spacings, which is past the edge of a 16- or 32-node grid.
The factor is also more than "1 − O(e^(−2r/ℓ))" for the high orders that n_theta allows: the variance is tanh(r/ℓ)^(2|m|), so for n_theta = 64 (as in the tutorial), m = 32 at r = 2ℓ keeps only 0.964^64 ≈ 10% of its variance. The physics argument for ℓ_r is fine, since it is grid-independent. But the docs should state that the default region of non-stationarity grows from ~1 to ~5 spacings, and that a fitted length_r now also sets how much m ≥ 1 structure the inner disk can carry.
There was a problem hiding this comment.
Updated docs/api/models/notes.md: taper is tanh(r/ℓ_r)^|m|, the default non-stationary region grows from ~1 to ~5 spacings, high orders recover slowly (n_θ=64, m=32 keeps ~10% at r=2ℓ_r), and a fitted ℓ_r sets inner m≥1 structure. 4aa810a
🤖 Addressed by Claude Code
| @@ -0,0 +1 @@ | |||
| `ToeplitzVisData(max_offset_mas=0.0)` is honoured (it fell back to the nuisance bound); `MultipoleDisk.suggest_n_r` warns when it steps down below the resolution it wants; the `MultipoleField` origin taper uses the radial correlation length `length_r` instead of the node spacing, so it no longer changes with `n_r`; already-projected `OIData` observables must be finite with non-negative errors | |||
There was a problem hiding this comment.
changelog.d/README.md has a default type for "changes to default behaviour" (see +outlier-mixture.default.md). The PR body lists four default changes, but this .fixed.md bullet presents them only as fixes. The one users will notice is the MultipoleField taper. "No longer changes with n_r" does not tell a reader that the default suppression radius grows from one node spacing to length_r (five spacings), so prior draws and every existing r_min_mas=0 MultipoleField fit change. The fragment also doesn't mention the new suggest_n_r UserWarning. Split out a +ip2b-small-fixes.default.md that states these consequences.
There was a problem hiding this comment.
Split into +ip2b-small-fixes.default.md stating the taper consequence (suppression radius one spacing → length_r, prior draws and fits change) and the suggest_n_r warning. 4aa810a
🤖 Addressed by Claude Code
| [onp.asarray(MultipoleField(zi, **kw).polar_values()) for zi in z] | ||
| ) | ||
| v = (onp.abs(onp.fft.fft(vals, axis=2)) ** 2).mean(axis=0) / n_theta | ||
| var[n_r] = (v[2 * (n_r - 1) // 4, m], v[-1, m]) # r = 4 and r = 8 |
There was a problem hiding this comment.
At r = 4 the test barely tells the fix apart from main. Expected is (tanh 2 / tanh 4)^6 ≈ 0.806, so rel=0.2 accepts [0.645, 0.967]. Main's spacing taper gives (tanh 4/tanh 8)^6 ≈ 0.996 (n_r = 9) and ≈ 1.0 (n_r = 17), only ~0.03 above the upper bound. The Monte-Carlo sd of a ratio of two 800-draw |FFT|² means is a few percent (~0.03–0.04 here), so main's result is within about 1 sd of passing. The test fails on main with seed 0, but another seed could let it pass.
Use the r = 1 node (index (n_r - 1) // 8: 1 and 2). There the expected ratio is (tanh 0.5 / tanh 4)^6 ≈ 0.0098, while main gives tanh(1)^6 ≈ 0.19 (n_r = 9) and tanh(2)^6 ≈ 0.80 (n_r = 17). That is a factor of 20–80 off, and the two grids also disagree with each other on main, which is the n_r-dependence this test means to catch.
There was a problem hiding this comment.
Moved to the r = 1 node: expected 0.0098 against main's 0.19 (n_r=9) and 0.80 (n_r=17). 4aa810a
🤖 Addressed by Claude Code
| f"suggest_n_r wants {wanted} radial nodes but the profile can " | ||
| f"build at most {n}: the Hankel quadrature may be under-" | ||
| "resolved on the longest baselines.", | ||
| stacklevel=2, |
There was a problem hiding this comment.
Minor. (1) If no rule above 16 builds, the loop exits at n = 16 without trying 16, so "can build at most 16" is claimed without being checked. (2) stacklevel=2 points at the caller, which is pipeline/disk.py:568 when the pipeline sizes nodes. The repo already has external_stacklevel() for this (used at oidata.py:331). (3) The docstring doesn't mention the new warning; one line would do.
There was a problem hiding this comment.
Reworded the warning so it no longer claims a checked maximum, uses external_stacklevel(), and the docstring mentions it. 4aa810a
🤖 Addressed by Claude Code
… and test, warning stacklevel, default changelog, shorter ToeplitzVisData init Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ear r=0 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
IP-2b of the 0.4.2 open-issues plan. Closes findings #604 F12, #604 F23, #604 D3 and #585#18.
Fixes (each test fails on main 6b80f75, passes here)
radio_toeplitz.py):max_offset_mas=0.0is honoured (is Nonetest, not falsiness). Test:test_a_zero_max_offset_is_honoured_....models.py):MultipoleDisk.suggest_n_rwarns when the rule steps down. Test:test_suggest_n_r_warns_when_it_steps_down.fields.py): theMultipoleFieldorigin taper istanh(r/length_r)^|m|, nottanh(r/spacing)^|m|. Born & Wolf'sa_m ~ r^|m|holds out to the scale on which the field varies, its radial correlation length. The node spacing is the grid's, so it changed withn_r. Derivation is in the code comment; docs sentence updated. Test: same physical field on n_r = 9 and 17 gives the same taper-variance ratio,(tanh(4/ell)/tanh(8/ell))^(2m). Two existing tests now setlength_rexplicitly to the old scale.oidata.py): already-projected observables (size != n_samples) skipped all validation. They must now be finite with finite non-negative errors, elseValueError(zero projected errors are still rejected where data are whitened). Tests:tests/test_oidata_projected.py.Changes to default behaviour
MultipoleFieldwithr_min_mas == 0: the taper scale changes from the node spacing tolength_r(default 5 spacings), so the m >= 1 harmonics are suppressed over a wider radius by default. Prior draws and any fit with a MultipoleField change.suggest_n_remits aUserWarningwhere it used to step down silently (return value unchanged).OIDatabuilt from projected data containing NaN/Inf/negative errors now raises (previously accepted).max_offset_mas=0.0now gives a zero shift bound instead of the nuisance default.Verified, not bugs
star.pypinv, zero diameter prior): not a bug.J^T J + diag(0, 1/s^2)is singular only when the diameter column of J is exactly zero; the diameter then decouples (P_ds = 0), so the null space is exactly the diameter axis andpinvgives the exact shape marginal. Script comparedpinvwith an inverse using a proper vanishing diameter prior on four cases (zero column, both zero, collinear ridge, generic): identical to 1e-16; min diagonal 0.0, never negative.OI_FLUX,T3AMP,VISAMP,VISPHI) take no fitted noise scale (inflated_errorsleaves them unchanged), so there is no scale parameter to carry Fisher information; their mean rows are in_whiten_with_log_norm(frozen=True). The quadraticsqrt2 log sigmahalf islikelihood.py, outside this PR's files (IP-2a area).test_closure.py:261/267: the tolerances bracket Monte-Carlo error of 4000 draws (relative sd of a variance is sqrt(2/4000) = 2.2%, so rtol 0.1 is 4.5 sigma; sd of a covariance entry is about 1/sqrt(4000) = 0.016 so atol 0.1 is 6 sigma). They are statistically justified but loose; tightening to about 4 sigma is IP-3's call.test_composition.py:449has a stated float32 reason.test_aliases.py:1415not examined further.sum < 1e-3bounds every term and no cancellation can hide an error;sum > 1e3for the wrong node can only pass if some term is large. The ordering assert intest_aliases.py:1128is weak but also cancellation-free. Test files, left to IP-3.Notes
notebooks/multipole_disk_rings.ipynb/docs/multipole_disk_rings.md(MultipoleField prior draws, a_m tracks) were rendered with the old taper (scale 1.29 mas; nowlength_r= 6 mas). The prose is updated; re-execute the notebook (OzSTAR if heavy) and runscripts/sync_tutorial_docs.pybefore release.test_latents_are_whitened_...now uses an annulus (the taper lowers variance near r = 0 by design; tolerance unchanged).test_kkt_report); Fix main red: LinearImage brightness is NaN at all-zero pixels, stopping L-BFGS-B (#662) #666 fixes it. Not touched here.Does any change pass by special-casing, tuning or loosening? No. No tolerance, seed or target was touched; the
length_rscale is derived from the field's own correlation length, and the new tests were shown to fail on main.🤖 Generated with Claude Code