[cailmdaley]

Closes #433. This adds the UNIONS catalogue paper's validation figures to the cosmo_val dashboard for the v2 catalogue. It is stacked on feat/cosmo-val-dashboard-figures.

Every figure below is rendered from #425's products: 10,094 tiles, 73.0M galaxies in the cut catalogue. All are systematics tests, so none emits ξ±, Cℓ or κ.

What's new

Code

• Catalogue-pass products: field_maps.py is extended and selection_hist.py added. One streamed pass over the cut catalogue and stars produces sufficient statistics for the SNR × size grid, star residuals, the r-magnitude map, α cells and bias regressions. A second pass over the comprehensive catalogue produces the magnitude histograms by selection stage.
• Position–shear products: position_shear.py (the TreeCorr NG estimator, with jackknife covariance and random subtraction) and correlations.py (the builder).
• Renderers: object_bias.py, selection_hist.py and tangential.py, plus new functions in psf.py, sky.py and weights.py. All are pure (products, path) -> PNG and registered in render.py, which gains a --correlations-dir flag.
• One command per snapshot: scripts/dashboard/build_dashboard.sh CATALOGUE_DIR RUN_DIR VERSION OUT_DIR [--gaia F] [--after JOBID] submits both builders in parallel and a render job that depends on them. --after chains it on a catalogue extraction.
  • Measured at 10k tiles: the products take 3m42s / 7.5 GB for galaxies and stars, and 59 min / 3 GB for the comprehensive pass. The correlations take about 20 min / 44 GB peak on 8 cores.
  • At 20.5k tiles I expect roughly 2× that (an extrapolation, recorded in the script headers).
• Gaia DR3 (G < 20, RUWE < 1.4): 17.45M stars covering all 20,515 tiles, staged once from GAVO's gaia.dr3lite by scripts/dashboard/stage_gaia.py. ESA's archive caps anonymous queries at 50k rows.
• Size-residual map: it now uses the paper's model normalisation, (T⋆ − T_PSF)/T_PSF.
• Schema: shapepipe_v2.yaml documents the star columns the code reads.
• Tests: two tests that already failed on the base branch are fixed: a ρ/τ driver fixture and schema non-columns.

Choices

• Star sample: "PSF validation stars" means both HSM flags = 0, the dashboard's ρ sample. ACCEPTED marks whether PSFEx used a star in its fit, so it isn't a reserve flag.
• Shapes: calibrated shapes with w_des, and no object-wise leakage correction, since raw E is v2's reference (Object-wise PSF-leakage correction zeros the same-object slope, not angular leakage; on v2 it raises τ0 by 52% #428).

Inventory of the paper's 32 figure files

Status	Paper figures
Already on the dashboard	n_eff and σ_e maps; e1/e2 maps; PSF FWHM map; star−PSF size map; ρ; τ; α–β–η posterior; ξ_sys/ξ+
Added here	SNR×size grid (weighted_bins); response histogram (galaxies only); residual star properties; star density map; r-magnitude map; magnitude histograms by selection stage; α per SNR×resolution cell; ξ_sys from α–β–η vs α only; additive-bias regressions; α(θ) in magnitude bins; galaxies around PSF stars; PSF stars around galaxies (λ); galaxies around tile centres; galaxies around Gaia stars (γ_t, γ_×)
Later	Galaxies around CCD centres; focal-plane mean shear; size–magnitude star selection; star response histogram. All need per-exposure galaxy positions or ShapePipe products not in the extracted catalogue. Affine per-CCD fits to stars give 0.27″ median residual, a possible route.
Not applicable	MCALens κ map (a separate pipeline, and cosmological signal); detection-image cutout

Figures and readings

SNR × size grid. Most galaxies sit at low SNR. The response rises from about 0.3 at SNR 10 to about 0.9, and shape noise is lowest for bright, small galaxies.

Galaxy response. R11 and R22 overlap and peak near 0.85; R12 and R21 peak at 0. Stars are omitted because v2 has no metacal star shapes.

Residual star properties. The ellipticity and fourth-moment residuals are centred on zero. The size residual is √(T⋆/T_PSF) − 1 = −0.3%: the PSF model is slightly too large.

Star density and size residual. Density rises toward the galactic plane. The size-residual map is mostly negative (median −0.58% in T) and has a positive south-west edge feature.

PSF FWHM and mean r magnitude. Mean galaxy magnitude anticorrelates with seeing, as the paper found for v1.

Selection stages. The final stage reproduces the cut catalogue's 73,023,642 rows. The v1 chip-defect, halo and pointing-count stages have no v2 cut and are omitted.

Object-wise α in 8×8 equal-count cells. α is about +0.03 for larger galaxies at SNR 20–40 and about −0.03 for the smallest; hatching marks |α| < 2σ.

α(θ) = τ₀/ρ₀ in magnitude tertiles.

• All galaxies: +0.002 to +0.006, matching the cached Consistent shear response, weights and additive bias (DES case d) #425 τ₀ and ρ₀ (within 0.14σ).
• The faint third (r > 24.1) leaks with α ≈ −0.03, trending toward zero at large θ. This is the same sign as the small-galaxy cells above, consistent with a PSF model that is too large over-deconvolving small, faint galaxies.
• DES-Y3 is omitted.
• Each bin uses its own shear response plus the global selection response. The faint bin's selection response is likely several times larger, so its |α| is probably underestimated by about 10–20%. The sign is safe. Per-bin selection responses need the sheared metacal branches of the comprehensive catalogue, and are left as a follow-up.

ξ_sys from the α–β–η model vs α only. The full model sits about 5× above τ₀²/ρ₀ at large θ and about 1000× at 1′, so v2's PSF systematic is dominated by the PSF-model-error term (β), not by leakage.

Additive-bias regressions, per 1σ of each property. Every trend is below 3.5×10⁻⁴ in e. The r-magnitude trend, about −3×10⁻⁴ in both components, is the largest at 5–8σ by fit errors that ignore spatially correlated noise.

Galaxies around 4.77M PSF stars. γ_t is slightly negative from 3′ to 30′ (about −1×10⁻⁵, χ²/ν = 1.45). This is the paper's radial excess again, weaker. γ_× is null.

PSF stars around galaxies (λ), source ellipticities mean-subtracted. The star − model residual is null (χ²/ν = 1.22 and 0.86). The PSF model has a small tangential signal (χ²/ν = 2.0). Its cross component is non-zero (χ²/ν = 10.3): a smooth trough of about −6×10⁻⁵ peaking near 20′, the tile and CCD scale.

Review checked whether this is a convention bug. It isn't: flipping g₂ doesn't remove it, and galaxy and star frames agree through τ₀ and the α₁/α₂ cells. A focal-plane-locked PSF pattern is the plausible source. It doesn't reach the shear, since λ₃ is null.

Galaxies around tile centres. Null (χ²/ν = 0.81 and 1.03).

Galaxies around 7.23M Gaia stars. Null in all three G bins, with randoms subtracted. χ²/ν of 0.3–1.1 suggests the jackknife errors are on the large side.

Review

An Opus code review found no estimator error that changes a conclusion. Its two numeric points are addressed above: λ now subtracts the mean ellipticity (it previously included the survey-mean PSF ellipticity), and the per-bin selection response is stated. Also fixed: reused jackknife patches are rebuilt when the catalogue grows, and the correlations job requests 128G at 20k tiles.

Testing

• Inside the CI image (sp_validation-ddd58fe0.sif): 572 passed, plus the schema test fixed on top. Ruff is clean.
• The merged branch renders 33 figures for Consistent shear response, weights and additive bias (DES case d) #425.
• build_dashboard.sh was dry-run against a stub sbatch to check the job graph.

---

Claude Opus 5.5 on behalf of Cail Daley

🤖 Generated with Claude Code