ngmix masked pixels: neighbours and defects as separate questions

[cailmdaley]

Every stamp we fit with ngmix has pixels we can't use as they are: light from a neighbouring object, or a defect such as a bad column, a cosmic ray, a bleed trail or a chip edge. Metacal makes these pixels matter more than their weights suggest. It shears the image in Fourier space, where a weight map can't enter, so each masked pixel's value is sheared and spread into its neighbours before the fit ever sees a weight. This PR sets out how we decide, pixel by pixel, what ngmix sees there, and when we drop an epoch altogether.

Two questions per pixel

The design rests on one separation. We ask two independent questions of each pixel: does it belong to a neighbour, and is it a defect? Each question has its own setting.

• Neighbours carry real light. A neighbour at a similar redshift is sheared much like the target, so how we treat its pixels is a choice about blending. BLEND_HANDLING makes that choice.
• Defects carry no sky information, so we have to fill them with something. DEFECT_FILL chooses the fill, and the epoch cuts decide when an epoch has too many defects, or one too close to the object, to be worth keeping.

We haven't yet chosen how to handle neighbours. We have a long-standing noise fill and UberSeg, the approach DES uses, and we will pick the default by comparing the two on real tiles and on blended simulations (#813). Because the defects are treated identically under both, that comparison changes only the neighbour pixels, and it measures blending and nothing else.

Pixel	BLEND_HANDLING = noisefill (default)	BLEND_HANDLING = uberseg
Neighbour footprint	zero weight, filled with noise	pixels nearer a neighbour than the target: zero weight, light left untouched
Beyond the tile edge	defect	defect
Defect (zero exposure weight, nonzero flag, invalid background RMS)	zero weight, filled by DEFECT_FILL	same

Neighbours

We find neighbours in the coadded tile. In each object's tile stamp, we mark with −1e30 every pixel that belongs to another object's footprint. SExtractor does this itself; for Stephen Gwyn's DR6 catalogue we take the footprints from his segmentation map. We apply that one mask to every exposure of the object.

• noisefill gives those pixels zero weight and replaces them with noise at the local background level.
• uberseg uses the segmentation map instead. We assign each pixel to its nearest footprint, and give zero weight to the pixels nearer a neighbour than the target. We leave their light as it is, because it is real sky that metacal shears along with the target. A noise-filled patch there would be a sharp edge in the image that does not shear with it. The workflow supplies the segmentation stamps separately.

The tile stamp is also −1e30 where it runs past the tile's edge. There the exposure is not blank: it holds the object's own light, cut off. So we treat these pixels as defects, not neighbours. We identify them as the unbroken runs of entirely −1e30 rows and columns that start at the stamp's border. On a simulated tile this matches the true off-tile region for 210 of 212 objects near the edge, to within the ±1 px uncertainty in where the stamp is centred. On a DR6 patch it matches the off-image region exactly for all 32 objects.

Defects

• noise replaces each defect pixel with noise at its background RMS.
• interpolate fills narrow defects by Clough–Tocher interpolation from the surrounding kept pixels, averaged over the stamp's four 90° rotations. "Narrow" means at most 3 px across with clean pixels on both sides: bad columns, 3 px bleeds, single pixels. Wider holes and edge bands get the noise fill. Interpolated pixels still carry zero weight, and a one-sided zero-weight hole pulls the model fit. For a column 8 px from a 0.5″ galaxy in one high-S/N epoch, it gives c₁ = −1.3e-3. So we also give zero weight to the three rotated copies of each interpolated pixel, keeping their light, and c₁ drops to 2e-6.

Which epochs we keep

We drop an exposure (epoch) from the fit when:

• a noise-filled defect pixel lies within EPOCH_CENTRAL_DEFECT_RADIUS (10 px) of the object's centre, or, under DEFECT_FILL = interpolate, an interpolated pixel lies within EPOCH_INTERPOLATED_DEFECT_RADIUS (7 px);
• more than EPOCH_MASKED_FRACTION_CUT (1/3) of the stamp is defect.

We veto central defects because a fill near the object biases its shape. Metacal shears a noise-filled hole in the galaxy's light, but not the sky around it. We chose each radius as the smallest at which columns, 3 px bleeds, single pixels and edge bands keep |m| < 1% and |c| < 5e-4 on galaxies of half-light radius 0.3″ and 0.5″ through a 0.7″ PSF. At 9 px, a noise-filled column on the 0.5″ galaxy already gives m₁ = −1.75% and c₁ = −1.1e-3. Interpolation disturbs the light less, so its radius is smaller. The bias matters because CFHT observes at a fixed orientation: detector-fixed defects like bad columns line up the same way in every epoch, so their biases add coherently rather than averaging away.

Neighbour pixels never enter these cuts. The neighbour mask comes from the tile, so it is the same in every epoch. A cut that counted it would drop every epoch of a blended object, and the object would vanish from the catalogue. We saw exactly that on a simulated tile of 21,851 objects when neighbours were counted as defects: the 10 px veto lost 31% of them. With this PR's cuts the same tile loses 89 objects (0.41%). Of those, 88 lie within 10 px of the tile edge, a strip the neighbouring tile also covers; we expect them to be measured there, but haven't checked. Two further simulated tiles lose none.

Open decisions

Simulations have no cosmic rays, bad columns or satellite trails, so their defect statistics can't settle these. The epoch-cut logs from 20–50 real tiles will; every run records how many epochs each cut drops.

1. The central-veto radius. On the same simulated tile the veto drops 123 of 64,662 epochs at 10 px. Most of the 89 lost objects are edge objects that the masked-fraction cut removes anyway.
2. The masked-fraction threshold: 1/3, or DES's 10%.
3. noise or interpolate as the default fill.

Technical notes

• Settings. BLEND_HANDLING (noisefill | uberseg), DEFECT_FILL (noise | interpolate), EPOCH_CENTRAL_DEFECT_RADIUS, EPOCH_INTERPOLATED_DEFECT_RADIUS, EPOCH_MASKED_FRACTION_CUT.
• What changes for noisefill. Before this PR, every −1e30 marker, neighbour or off-tile, counted towards a single 1/3 masked-fraction cut, and there was no central veto. Now the cut also counts zero-weight and bad-RMS pixels and ignores neighbour markers, and the central veto is on by default. Under the default noise fill, each kept epoch's weights and filled pixels are unchanged, and a test pins this. Which epochs we keep does change, so both arms of the noisefill/UberSeg comparison have to run on this code, not against an older catalogue.
• Known approximation. A neighbour footprint that fully covers the row or column next to an off-tile band extends the band, and we read its pixels as off-tile.
• Not included. Symmetrising neighbour weights (A/B: 4-fold weight symmetrisation — shape impact and S/N cost #815) and subtracting neighbours with a multi-object fit (MOF) remain options to compare later. We tried and rejected a symmetrised defect fill: it quadruples the filled area near the object, and m for a 3 px bleed 10 px from a 0.5″ galaxy goes from −0.64% to −2.7%. The code is kept on feat/symmetrized-defect-fill.
• Evidence for the epoch-cut numbers: /n17data/cdaley/scratch/dr6seg/veto_cost/ on candide.

Supersedes #915 and #916. Part of #602.

🤖 Generated with Claude Code