https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-290 DRP 2-D Pipeline <p>We need to be able to calibrate all fibers on a CCD by calibrating just one, so the fiber flats need to preserve the fiber-to-fiber throughput variations, whereas currently they don't.</p> <p>In the process, remove <tt>DetectorMap._throughput</tt>: throughput variations will be taken care of in the flat-field, and that also allows us to correct throughput as a function of wavelength rather than a single value per fiber.</p> PIPE2D-290

Fix normalisation of fibers in flats

Story Normal Done Done price price Thu, 13 Sep 2018 18:17:01 +0000 Thu, 18 Oct 2018 15:40:41 +0000 Tue, 16 Oct 2018 20:07:23 +0000 0 3 <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a>, are you available to take a look at this, please?</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> price@neverland:~/pfs/drp_stella (tickets/PIPE2D-290=) $ git sub commit 4415c44ee44de4e2ef7f8b53e6601c5be889187d Author: Paul Price &lt;price@astro.princeton.edu&gt; Date: Thu Sep 13 14:32:15 2018 -0400 remove SWIG bindings We're using pybind11 now. How have these managed to stick around so <span class="code-object">long</span>? python/pfs/drp/stella/stellaLib.i | 152 -------------------------------------- 1 file changed, 152 deletions(-) commit 80bdd284fd72c1c9e1e6337415e0bf9660cdd54f Author: Paul Price &lt;price@astro.princeton.edu&gt; Date: Thu Sep 13 14:53:40 2018 -0400 DetectorMap: strip throughput A single value per fiber isn't sufficient to characterise the throughput as a function of wavelength <span class="code-keyword">for</span> each fiber. We'll <span class="code-keyword">do</span> <span class="code-keyword">this</span> in the flat-field instead of carrying it around here. include/pfs/drp/stella/DetectorMap.h | 25 +--------- python/pfs/drp/stella/constructFiberTraceTask.py | 62 ------------------------ python/pfs/drp/stella/detectorMap.cc | 27 +++-------- python/pfs/drp/stella/detectorMapContinued.py | 16 +----- python/pfs/drp/stella/extractSpectraTask.py | 1 - src/DetectorMap.cc | 45 +---------------- tests/test_DetectorMap.py | 32 ++++-------- 7 files changed, 20 insertions(+), 188 deletions(-) commit 0b76eff8b900b509d7250d98e3c08414b1c383ee Author: Paul Price &lt;price@astro.princeton.edu&gt; Date: Tue Oct 2 21:34:44 2018 +0000 FiberTraceSet: add extractSpectra method That<span class="code-quote">'s why you have a FiberTrace, right? So let'</span>s add a method to <span class="code-keyword">do</span> that. python/pfs/drp/stella/fiberTracesContinued.py | 29 +++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) commit 7ccc5150684ca409f821866d3f1625ecb8362c5a Author: Paul Price &lt;price@astro.princeton.edu&gt; Date: Tue Oct 2 21:37:28 2018 +0000 refactor flat field normalisation Previously, each fiber was independently normalised, which means that flux calibrating one doesn't allow you to flux calibrate anything <span class="code-keyword">else</span> (well, we have a throughput measurement in the DetectorMap, but that's not a function of wavelength). Instead, normalise all fibers to the mean spectrum of the fibers. This keeps the fibers tied together. python/pfs/drp/stella/constructFiberFlatTask.py | 115 +++++++++++++----------- 1 file changed, 61 insertions(+), 54 deletions(-) commit 2801f730a15e1ab95cc21f169c174ad259f129de Author: Paul Price &lt;price@astro.princeton.edu&gt; Date: Tue Oct 2 21:36:06 2018 -0400 make some files flake8-clean python/pfs/drp/stella/constructFiberTraceTask.py | 2 -- python/pfs/drp/stella/detectorMapContinued.py | 3 +-- python/pfs/drp/stella/spectraContinued.py | 4 ++-- 3 files changed, 3 insertions(+), 6 deletions(-) </pre> </div></div> <p>I'm hoping to discuss this with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> later on today. </p> <p>What is the advantage of calibrating all by calibrating one? Besides that I guess that for flats (fiber throughput) I'd think of them as 600 separate cameras. At least until we learn better.</p> <p>Thanks for the pushback, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> &#8212; it encouraged me to go and verify that what I'm doing actually makes sense. I tried flattening the flats, and seeing how they turned out. There wasn't much improvement, which forced me to get in and fix it up. Now it's a lot better, and I think the plots demonstrate why this is important. Here's the extracted spectra from a flat after flattening with the 4.0 release:<br/> <span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/11501/11501_before-5694.png" style="border: 0px solid black" /></span><br/> Notice that the spectra from the different fibers don't follow each other exactly. The standard deviation from summing the flux over rows 1000 to 3000 is 0.0084 mag (that's a raw sum; we could calculate this statistic as a function of wavelength, but we just want a value for comparison purposes).<br/> Here's the same spectra after flattening with this submission:<br/> <span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/11500/11500_after-5694.png" style="border: 0px solid black" /></span><br/> The spectra are now much more closely aligned, and the standard deviation over the same aperture is 0.0013 mag. We have, in effect, calibrated all the fibers against a single spectrum (viz., a continuum fit to the mean flat-field). Now the flux calibration problem is reduced to calibrating that spectrum against some source with a known spectrum in physical units.</p> <p>Merged to master after receiving approval on GitHub.</p> Relates PIPE2D-298 Development Rank 0|s0011k: Reviewers hassan Story Points 5.0