https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-736 DRP 2-D Pipeline <p>Normalization in the 2d subtraction seems to off. For instance, for the example in <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-735" title="Offset in x direction with 2d subtraction" class="issue-link" data-issue-key="PIPE2D-735"><del>PIPE2D-735</del></a>, the flux in the whole image is 65K, while in the subtracted image the whole flux is still 14K, i.e., it is very far from 0. The figure is showing what fraction of flux is still in the residual image, after subtraction. As we can see, for some fibers more than 30% of flux is still present after subtraction. Possibly this is due to poor centering, but I believe I saw this issue also before.</p> <p>Robert also mentioned that we saw some weird stuff with normalization, but I am not sure if this is connected.</p> <p> </p> PIPE2D-736

Normalization in 2d subtraction

Task Normal Won't Fix Won't Fix Unassigned ncaplar Mon, 22 Feb 2021 21:28:42 +0000 Fri, 12 Mar 2021 13:28:33 +0000 Fri, 12 Mar 2021 13:28:33 +0000 0 2 <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a> Let me just verify quickly if that is still a problem, it does seems to be that it changes quite a lot from one run to another (which had different qualities of the centroid)</p> <p>Updated after fix introduced in <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-726" title="Regenerate detectorMap shifted in spatial direction for SuNSS" class="issue-link" data-issue-key="PIPE2D-726"><del>PIPE2D-726</del></a>. Residuals are now on the order of 3% which I presume is on the order of what I would expect. I will investigate if that is true.</p> <p>I meant to comment that the centroiding errors would mess up the normalisation; sorry.</p> <p>Are the 3% errors coming from the normalisation? We should be able to get the right at a level that doesn't affect sky subtraction; or is the 3% coming from the state of your PSF modelling? Or the photon noise?</p> <p>Ok, the ``effect'' is due to two factors, first one being dominant.</p> <p> </p> <p>1. The dominant effect: The pipeline (here: <a href="https://github.com/Subaru-PFS/drp_stella/blob/32139567407954633cfc6d0a9bac1a32776640db/python/pfs/drp/stella/subtractSky2d.py#L268)" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/drp_stella/blob/32139567407954633cfc6d0a9bac1a32776640db/python/pfs/drp/stella/subtractSky2d.py#L268)</a> normalizes by:<br/> a) computing sum(psf_model**2)<br/> b) computing sum(psf_model*image)<br/> c) computing renormalizing value as sum(psf_model**2)/sum(psf_model*image)<br/> d) psf_model_renormalized=psf_model * value </p> <p>As we can see, this does not ensure that sum(psf_model_renormalized)==sum(image), as I expected (unless psf_model_renormalized = constant * image). </p> <p>When the difference between pfs_model and image is larger, due to bad centering or bad modeling, the ``effect'' increases. </p> <p>2. Less important: I have been reporting values from 14x14 array around the psf cut, while the pipeline normalizes in square given by size of input psf, i.e., 21x21 pixels. However, that is a subdominant effect to the first one.</p> <p> </p> <p>The reason for this normalization is now understood. This is an optimal solution for the images dominated by noise errors (e.g., faint galaxies in HSC). In the future we might investigate if this is an optimal solution for this use case. At the moment closing this with Won't Fix and possibly reopening this discussion in the future.</p> Development Rank 0|zzs47y:zzw Sprint 2DDRP-2021 A3