https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1143 DRP 2-D Pipeline <p>Reducing the data for faint galaxies (visit=83244..83249) taken during the Nov run, I just noticed that the sky subtraction for some fibers is (still) significantly bad. Commands I ran are in this <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15492/15492_reduction_run08_galaxies_example.sh" title="reduction_run08_galaxies_example.sh attached to PIPE2D-1143">reduction_run08_galaxies_example.sh^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span>. Some examples of good and bad subtraction are shown below. Those are coadded pfsObjects but I see the same things in pfsMerged and in a single visit.</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/15491/15491_bad_subtraction_example1.png" style="border: 0px solid black" /></span></p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/15490/15490_good_subtraction_example1.png" style="border: 0px solid black" /></span></p> PIPE2D-1143

sky subtraction is bad for a certain fraction of fibers in the faint galaxy frames taken during Nov run

Task Major Done Done price Kiyoto Yabe collab-datarelease Fri, 13 Jan 2023 14:19:48 +0000 Fri, 17 Feb 2023 04:14:34 +0000 Thu, 16 Feb 2023 16:09:54 +0000 0 5 <p>Could you please tell me how the sky fiber positions were selected?</p> <p>Sky fibers used for the subtraction? I didn't specify them so config.selectSky.fiberFilter='ALL'.</p> <p>I thought that this was a "proper" extra-Galactic field from HSC, so shouldn't we have SKY objects? Will we have SKY objects in the next engineering runs?</p> <p>I don't think that this can explain the problem, though, as we are over subtracting the OH.</p> <p>Murata-san's sky object DB was used and we have 500 SKY fibers (in entire focal plane not SM1&amp;3) in this field.</p> <p>Ah, OK. So they were specified in the pfsDesign, which is all that matters. </p> <p>You shouldn't ever need to <tt>config.selectSky.fiberFilter</tt> except for interactive fiddling (a better solution is to write a pfsConfig to the rerun).</p> <p>FWIW, attached is the location of fibers with large residual (using an isolated sky line) and sky fibers on PFI plane. It looks that there is no strong (anti-) correlation?</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15494/_thumb_15494.png" style="border: 0px solid black" /></span></p> <p>This is due to fibers being in the penumbra of nearby black spots. Based on a bunch of sky lines from two exposures (83149 and 83244), the throughput appears to be linear with distance from the black spot (slope of 1.232/mm) up to a distance of 1.108 mm.<br/> I've added a correction for this effect in <tt>reduceExposure</tt>. The correction modifies the normalisation, so that the flux is still what is extracted from the image.<br/> In order to keep track of the correction (and other reduction operations and quality measurements), I've added a new element to the spectra, currently called "notes". This is an addition to the datamodel, so needs approval via a DAMD ticket.<br/> Using the fixed code, I get the following sky-subtracted merged spectrum for <tt>visit=83244 fiberId=478</tt> (which observes <tt>objId=43158588759244917</tt>, which had the bad sky subtraction):</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/15501/15501_Xnip2023-01-20_17-15-59.jpg" width="1000" style="border: 0px solid black" /></span></p> <p>This includes the changes of <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-145" title="Add facility to hold history/metrics of spectra" class="issue-link" data-issue-key="DAMD-145"><del>DAMD-145</del></a> (adding reduction notes to spectral products) in order to record the values that were used.</p> <p>Changes approved. Could not see any obvious issues with the proposed changes, including the new <tt>notes</tt> mechanism. Perhaps a slope to model the penumbra may be too simplistic for the future, but it works now and is a good start.</p> <p>Need to check: it looks like I might be making the correction twice (once in extractSpectra, and once in reduceExposure).</p> <p>I double-checked, and I am <b>not</b> making the correction twice. The second correction is in <tt>pfs.drp.stella.pipelines.extractSpectra</tt> (part of the Gen3 pipeline that replaces <tt>reduceExposure</tt>), not <tt>pfs.drp.stella.extractSpectra</tt> (the building block).</p> <p>Merged to master.</p> Blocks PIPE2D-1140 DAMD-145 Relates PIPE2D-1165 PIPE2D-1140 Development Rank 0|02qph7:07fi Reviewers hassan Sprint 2DDRP-2023 A