https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1156 DRP 2-D Pipeline <p>By looking at the original data I noticed that in noisy parts of the spectra some isolated pixels remain unflagged, and with values that do not seem to be coherent with the (current) noise estimate of the spectrum. These pixels are used by PIPE1D just as others and can induce wrong measurement if their value is not correct. This problem might be solved by the correction of the noise estimate though. <span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/15514/15514_NovRun_mask_621_635nm.png" style="border: 0px solid black" /></span></p> PIPE2D-1156

fluxTable masks too severe

Bug Normal Done Done price vlebrun Wed, 1 Feb 2023 12:17:47 +0000 Wed, 1 Mar 2023 18:36:10 +0000 Wed, 1 Mar 2023 18:36:10 +0000 0 7 <p>Could you please provide instructions to reproduce this?</p> <p>I used Topcat on the FluxTable extension, plotted all the pixels (in red), then selected all pixels with mask=0 and overplotteed them (in blue)</p> <p>the file is pfsObject-00010-00001-1,1-009954820000a57e-006-0x6e85dbfdd0e3cf63.fits</p> <p>from the GE_COS1 dataset (pfsDesignId=0x608f5cd5e916bf12: visit=83244..83249 )</p> <p>Actually I had no clear expectation on such a bad part of the spectrum, an I'd wait for the issues on the noise evaluation before focusing on this one, it might just be that it will be solved then</p> <p>On what file? How was the data processed? What version was used? What do you expect the result to be, and how does this differ from that?</p> <p><a href="https://github.com/Subaru-PFS/drp_doc/blob/master/pipelineUser/help.rst" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/drp_doc/blob/master/pipelineUser/help.rst</a></p> <p>At least part of the problem is that <tt>mask=0</tt> excludes good pixels as well as bad. On this coadd, we've <tt>OR</tt>-ed the input masks while excluding bad pixels, so <tt>CR</tt> means that there was a cosmic-ray at that pixel in one of the inputs, but we've excluded it in the process of coadding (but maybe we didn't do a great job with neighbouring pixels). We've also added a mask plane (which, due to a bug, wasn't showing up; and wasn't being calculated very well) called <tt>OVERLAP</tt> which indicates pixels from one arm that have better coverage by another arm. So you should ignore pixels with <tt>(mask &amp; NO_DATA) != 0</tt>. </p> <p>I've fixed a couple of things in the fluxTable, and made the plot with the correct masking, and I think it looks much better.</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/15543/15543_pip2d-1156.png" width="1000" style="border: 0px solid black" /></span></p> <p>looks definitely better indeed. Is this new masking ( ignore pixels with <tt>(mask &amp; NO_DATA) != 0 ) going to be definite, or just to wait for your fix ?</tt></p> <p>In general, you should never expect that all mask planes indicate something bad, so each time you use a mask you need to ask what mask planes you care about.</p> <p>Merged.</p> Development Rank 0|02qph7:07f Sprint 2DDRP-2023 A