[xoubish]

In addition to reading DR1, there are new plots and more description.

[xoubish]

@jkrick I couldnt add a reviewer so here's a comment and thank you. These are changes made to Q1 notebooks to work with DR1. There are some cleanups and better plots also.

[jkrick]

A combo of my thoughts and AI review. Probably not all of the AI found things are important to do. I'll approve now, I do not need to review these notebooks again.

Review of PR 398 (Euclid DR1 MER notebooks 1 and 2)

Both notebooks ran end to end against irsadev with no errors (notebook 1: ~22 min, notebook 2: ~3 min).

Blocker

1. TODOs that are release-day steps for when DR1 goes live are fine to keep: switching from irsadev to production, the access_url redirect, the Kümmel et al. reference and DR1 DOI, and re-checking the position and plot limits against production. The open questions and "if IRSA adds X" notes below should be resolved, and their TODOs removed, before merge. They are all in notebook 2 (2_Euclid_intro_MER_catalog.md). Most are HTML comments (<!-- TODO ... -->), so they don't show in the rendered notebook, only in the source; search for TODO:

   • line 71: mention DR1 morphology and cutouts tables if IRSA adds them
   • line 86: confirm with IRSA where the deep visits and the EMDS/HSC fields are, and what the ..._photometry tables hold (see #9)
   • line 170: flux_radius, sersic_visnir_flags, sersic_ext_flags missing from the irsadev tables; ask IRSA whether they will be added
   • line 265: describe the bits of det_quality_flag and the flag_<band> columns once the DR1 data product description is available
   • line 324: check for a DR1 MER Photometry Cookbook, and confirm with MER whether they recommend the template-fitting or the aperture recipe for DR1
   • line 491: mention the filling_factor column if IRSA adds it

   For the ones that depend on IRSA adding columns or tables (71, 170, 491), try to get an answer now.

Likely crashes or wrong results

2. Notebook 1, line 235: picking the wide tile can crash. The code takes the first "official" row with no check, and "official" means the text contains -DATA_RELEASE- with dashes, while the prose (line 87) only promises DATA_RELEASE. If nothing matches, [0] raises IndexError and every later wide-vs-deep cell fails. This is closer than it looks: in this run only 1 of the 21 wide VIS mosaics is official (see #13).

3. Notebook 1, line 343: the code admits a file in the search results may be missing from the bucket, but only the wide header reads handle that. Making cutouts for the deep bands (and opening the full deep VIS image at line 249) has no FileNotFoundError handling. One missing band breaks the RGB, comparison and source-extraction sections.

4. Notebook 2, line 357 (and line 436): the colors divide by flux_vis_to_X_templfit, which is never checked to be positive. This happens in the real data: in this run 1-2 of the 2964 stars have flux_vis_to_{y,j,h}_templfit <= 0. One bad row gives NaN, and np.argmin returns the first NaN, so all three picks can be the same star. The color-color selection at line 436 only checks flux_X_templfit, so these stars also make NaN colors there. Check the flux_vis_to_X_templfit columns as well.

5. Notebook 2, line 328: flux_detection_total is treated as VIS flux, which is only true for VIS-detected sources. vis_det is queried but never used. In this run it doesn't change anything: all 2964 stars have vis_det = 1, and none of the 82k NIR-detected sources has point_like_flag = 1. It's a hidden risk for anyone who reuses total_mag on non-star samples, though. Filter on vis_det == 1 (or say in the docstring that it requires VIS-detected sources), which also uses the column the query already fetches.

6. Notebook 2, line 378: the text (line 151) tells users they can switch to any of the four catalog tables, but vis_cutout always searches the wide mosaic collection, and the band list assumes DECam. Switching to a deep table gives tile IDs that never match (IndexError). A northern field has no DECam g flux, so no star qualifies and the empty argmin raises ValueError. Either qualify the sentence at line 151 or choose the SIA collection from the table/tile ID prefix (see also #14).

Text, clarity and consistency

7. MER is never defined in either notebook; the only hint is the reference title "The MERge processing". Define it at first use in the intro of each notebook (notebook 1 line 68, notebook 2 line 64). For example: "MER (the Merge processing function) combines the VIS, NISP and ground-based images into mosaics on a common grid, and detects and measures sources on them to build the catalog."

8. Notebook 2, lines 65 and 366: "(see the MER mosaics notebook)" should be a link, e.g. [MER mosaics notebook](1_Euclid_intro_MER_images.md) (the same relative-link style the HATS notebooks use).

9. The wide/deep descriptions don't fully match between the two notebooks, and part of the gap is parked in an internal TODO:

   • Notebook 1 (line 82) says deep tiles cover "the deep fields", which implies only EDF-N/S/Fornax. Notebook 2 (lines 78, 83) says deep tiling is used for any field too deep for wide tiles, including COSMOS, CDFS and AEGIS (deep_mode). Notebook 1 should match this.
   • The images come as 2 collections (wide and deep tiling), but the catalog is split into 4 tables (survey/mode x wide/deep). Neither notebook maps one onto the other. Add one sentence to each, e.g. "wide_survey and wide_mode sources are on the wide mosaics; deep_survey and deep_mode sources are on the deep mosaics." This is also the root of #6.
   • Notebook 1 (lines 85-87) explains that wide tiles in the deep fields have several mosaics per band (one official, the rest single-pass "deep visits", told apart by PPLANID). Notebook 2 doesn't mention this. Its internal TODO (lines 86-92) says it's not yet confirmed which catalog table the deep visits went into, or where the EMDS/HSC fields ended up. Please resolve this TODO with IRSA before merge, then describe the deep visits in notebook 2 consistently with notebook 1 and cross-link. Same TODO: describe the two extra tables seen on irsadev (euclid_dr1_mer_catalogue_deep_survey_photometry, ..._deep_mode_photometry), or say why they're left out. Note that Irsa.list_catalogs in this run lists only the 4 + Q1 tables, so they may no longer exist.
   • Only notebook 1 gives the depth of deep tiles (about 8x the VIS exposure) and says wide tiles use "the same tiling as in Q1". Notebook 2 points to Table 5 of Kümmel et al. instead. Neither is wrong, but consider reusing the same short paragraph in both.

10. notebook_metadata.yml, line 12: the gallery card still says notebook 2 makes a "color-magnitude diagram", but it now makes a color-color diagram. tutorials/euclid/euclid.md also still describes only Q1.

11. Notebook 1, line 454: the comment says the background mesh is "larger than our galaxy", but it isn't. bw=bh=128 px is 12.8", while the galaxy at the center of the cutout is about 250 px (25") long (see the sep plot). The background map shows a bump under the galaxy, so some galaxy light is being subtracted as background. Either use a larger mesh (e.g. 256-300 px, half the 600 px cutout) or fix the comment. Also, the text never says the target is a galaxy; it calls it "a position in the Euclid Deep Field North" (line 138). Say what the object is (it's a bright edge-on spiral), since every plot is centered on it.

12. Notebook 1, line 272: "This reads the full image from the cloud, so it may take a minute." The full notebook took ~22 min here, vs ~3 min for notebook 2. Separately, the deep VIS image is already loaded in full, yet the VIS cutout in section 6 reads the same 0.42 GB file from S3 a second time. Cutting from im_mer_irsa would save time. Worth a note in the Data volume section on the expected run time. Also, the full mosaic is opened at line 252 with hdu_mer_irsa = fits.open(...) and never closed, unlike make_cutout (line 332), which uses a with block. The handle has to stay open across cells, so a with doesn't fit here; add hdu_mer_irsa.close() once im_mer_irsa and header_mer_irsa have been extracted.

Things the run showed that need an explanation in the text

13. Notebook 1, section 3: the two wide tiles behave differently. Tile 102158276 has 10 VIS mosaics and none of them is official (all DATA_RELEASE_DEEP); tile 102158277 has 11, with one official. The prose (line 86, "Only one is the official wide survey mosaic") reads as if every wide tile in the deep field has one. Either the official mosaic for 102158276 is missing from IRSA (ask IRSA), or the text should say a tile may have none in this collection. Because only one official mosaic exists, the old concern that "reruns could pick a different wide tile" doesn't apply at this position. The fix for #2 still applies.

14. Notebook 1, section 3: the PPLANID values of the single-pass mosaics come from two pipeline versions (2.0.25-...-PLAN-000000 and 2.0.32-...-PLAN-000001). One mosaic per tile is the 2.0.32 version. If that's a reprocessing of the same pass, then "Wide (single pass 1..10)" in the comparison plot double-counts one pass. Worth checking with IRSA/MER, and adding a sentence about it.

15. Notebook 1, comparison plot (section 7): the official wide mosaic is noisier (2.7e-3) than every single-pass mosaic (2.3-2.6e-3), and it has a satellite trail across the top. Readers will wonder why the "official" one looks worse. Add a sentence saying the official wide mosaic is itself a single wide-survey visit, so similar noise is expected, and that the trail is a known artifact.

Smaller issues

16. Notebook 2, line 289: .filled(0) assumes point_like_flag is a masked column. In this run it is (and masked for all but ~3000 rows), so it works. Elsewhere the notebook uses np.ma.filled(...), which also works on a plain column; use that here too for consistency.

17. Notebook 1, section 7 cutout grid: the bands come out in server order (Y, H, J, VIS, I2, R2, G, Z), not by wavelength, and the HSC labels are long ("Hyper Suprime-Cam_I2"). Sort the grid by wavelength and shorten the labels (e.g. "HSC i").

18. Notebook 1, line 253: print(hdu_mer_irsa.info()) prints the table and then None. Drop the print.

19. Notebook 2: the sky-map (line 252) and color-color (line 466) cells end with an ax.set_... call, so the notebook shows stray Text(...) output lines. End them with plt.show() or a ;.

20. Notebook 2, section 5: the VIS cutouts are 3" (30 x 30 px). All three stars look identical and show nothing around them. A larger size (e.g. 10") would make the cutouts more informative.