[PIPE2D-736] Normalization in 2d subtraction Created: 23/Feb/21  Updated: 12/Mar/21  Resolved: 12/Mar/21

Status: Won't Fix
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: ncaplar Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File residual_normalization_after_726.png     PNG File residual normalization.png    
Sprint: 2DDRP-2021 A3

 Description   

Normalization in the 2d subtraction seems to off. For instance, for the example in PIPE2D-735, the flux in the whole image is 65K, while in the subtracted image the whole flux is still 14K, i.e., it is very far from 0. The figure is showing what fraction of flux is still in the residual image, after subtraction. As we can see, for some fibers more than 30% of flux is still present after subtraction. Possibly this is due to poor centering, but I believe I saw this issue also before.

Robert also mentioned that we saw some weird stuff with normalization, but I am not sure if this is connected.

 



 Comments   
Comment by ncaplar [ 24/Feb/21 ]

hassan Let me just verify quickly if that is still a problem, it does seems to be that it changes quite a lot from one run to another (which had different qualities of the centroid)

Comment by ncaplar [ 24/Feb/21 ]

Updated after fix introduced in PIPE2D-726. Residuals are now on the order of 3% which I presume is on the order of what I would expect. I will investigate if that is true.

Comment by rhl [ 25/Feb/21 ]

I meant to comment that the centroiding errors would mess up the normalisation; sorry.

Are the 3% errors coming from the normalisation? We should be able to get the right at a level that doesn't affect sky subtraction; or is the 3% coming from the state of your PSF modelling? Or the photon noise?

Comment by ncaplar [ 25/Feb/21 ]

Ok, the ``effect'' is due to two factors, first one being dominant.

 

1. The dominant effect: The pipeline (here: https://github.com/Subaru-PFS/drp_stella/blob/32139567407954633cfc6d0a9bac1a32776640db/python/pfs/drp/stella/subtractSky2d.py#L268) normalizes by:
a) computing sum(psf_model**2)
b) computing sum(psf_model*image)
c) computing renormalizing value as sum(psf_model**2)/sum(psf_model*image)
d) psf_model_renormalized=psf_model * value 

As we can see, this does not ensure that sum(psf_model_renormalized)==sum(image), as I expected (unless psf_model_renormalized = constant * image). 

When the difference between pfs_model and image is larger, due to bad centering or bad modeling, the ``effect'' increases. 

2. Less important: I have been reporting values from 14x14 array around the psf cut, while the pipeline normalizes in square given by size of input psf, i.e., 21x21 pixels. However, that is a subdominant effect to the first one.

 

Comment by ncaplar [ 12/Mar/21 ]

The reason for this normalization is now understood. This is an optimal solution for the images dominated by noise errors (e.g., faint galaxies in HSC). In the future we might investigate if this is an optimal solution for this use case. At the moment closing this with Won't Fix and possibly reopening this discussion in the future.

Generated at Sat Feb 10 15:57:09 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.