https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-796 DRP 2-D Pipeline <p><a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-785" title="Add option to use tweaked detectorMap to define fiberProfiles" class="issue-link" data-issue-key="PIPE2D-785"><del>PIPE2D-785</del></a> introduced an affine transformation to the detectorMap trace centers when building fiber profiles. This should be generalised so it can be used on any exposure before running <tt>profiles.makeFiberTracesFromDetectorMap</tt>. We should also persist these important results by some means to make things more repeatable.</p> PIPE2D-796

Allow adjustment of detectorMap in reduceExposure

Story Normal Done Done price price Fri, 26 Mar 2021 15:09:02 +0000 Tue, 4 May 2021 17:39:49 +0000 Tue, 4 May 2021 17:39:49 +0000 0 3 <p>I may be missing something, but at present I think that the only way to update a <tt>DetectorMap</tt> is with <tt>reduceArc</tt> (I may be missing some bootstrap commands).  I think we need to generalise that to allow a combination of arcs and continuum exposures (quartz or moonlight), as especially in the blue we don't have enough lines to define the location of the traces very well.</p> <p>As I've probably written in a different ticket, with a global distortion model, even featureless spectra probably define the spectral solutions pretty well, in a differential sense.  Obviously we'll use emission lines when we can.</p> <p>Looking at the SuNSS data I'm seeing strange patterns in the x (but not y) residuals. I'm wondering if these are due to the continuum in some of the exposures coming from moonlight (or, for the diffuse fibres, possibly a bright object in the field). Upon setting <tt>reduceExposure.doSubtractContinuum=True</tt> to see if this was indeed the problem, I noticed that the sky subtraction wasn't very good for some of the exposures due to the location of the traces being wrong. This could/should be handled by updating the DetectorMap, but as we don't know the cadence on which this will be possible the availability of the low-order tweaks provided by this ticket will significantly improve the sky subtraction.<br/> I do not yet know if the continuum is the source of the problems that I'm seeing.</p> <p>While you're in the code, is there a reason why <tt>FitDifferentialDetectorMapTask.run</tt> takes four arguments unpacked from <tt>oldDetMap</tt> instead of the old <tt>DetectorMap</tt> itself? And it appears to want a <tt>VisitInfo</tt> purely for the sake of a random seed (is that right?). </p> <p>And please add a <tt>_DefaultName</tt> to <tt>FitDifferentialDetectorMapTask</tt> and all other PFS tasks that don't have one.</p> <p>See also <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-809" title="Update DetectorMap from a combination of arcs and flats" class="issue-link" data-issue-key="PIPE2D-809"><del>PIPE2D-809</del></a></p> <p>I've pushed a quick hack (no proper config etc.) to <tt>u/rhl/<a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-796" title="Allow adjustment of detectorMap in reduceExposure" class="issue-link" data-issue-key="PIPE2D-796"><del>PIPE2D-796</del></a></tt> </p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=rhl" class="user-hover" rel="rhl">rhl</a> I don't see anything in your branch that corresponds to the goal of this ticket. Are you wanting me to incorporate your work? Otherwise, I think you should put it on a separate ticket for review and merge.</p> <p>Sorry, I take it back. I see something relevant in the commit labelled "Hack hack hack".</p> <p>Paul asked me to add a note about exactly what I'm looking for in the related tickets <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-785" title="Add option to use tweaked detectorMap to define fiberProfiles" class="issue-link" data-issue-key="PIPE2D-785"><del>PIPE2D-785</del></a>, <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-796" title="Allow adjustment of detectorMap in reduceExposure" class="issue-link" data-issue-key="PIPE2D-796"><del>PIPE2D-796</del></a>, and <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-809" title="Update DetectorMap from a combination of arcs and flats" class="issue-link" data-issue-key="PIPE2D-809"><del>PIPE2D-809</del></a>.</p> <p>I'd like to use the global properties of the DetectorMap wherever possible, so whenever we need accurate positions of spectral features (in x and/or y) I'd like to see the workflow being:</p> <ol> <li>tweak the DetectorMap if necessary </li> <li>use the DetectorMap as the truth.</li> </ol> <p>In some cases this may require extra or repeated image processing (<em>e.g.</em> <tt>reduceExposure</tt> needs to run <tt>reduceArc</tt> to update the detector map, and possibly part of <tt>ConstructFiberProfilesTask</tt> too).</p> <p>That's why I expected <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-785" title="Add option to use tweaked detectorMap to define fiberProfiles" class="issue-link" data-issue-key="PIPE2D-785"><del>PIPE2D-785</del></a> to be part of the solution to the other two tickets, although I am not sure that it's worth going back and redoing the work at this point.</p> <p>Because the DetectorMap is fundamentally about <tt>(x, y)</tt> the tweak code should be based on <tt>(x0, y0, x, y)</tt> rather than <tt>(x0, lambda0, x, lambda)</tt>, which I believe to be the current API and one reason I didn't address this ticket myself. If we also include the errors in the positions <tt>(dx, dy)</tt> we should be able to use the same code to handle arcs (with small dx and small dy) and fibres (with large dy).</p> <p>Finally ready for review; sorry about the delays.</p> <p>Quite a few big changes, but changes look OK to me. No specific comments to address.</p> <p>Merged.</p> Development Rank 0|02qpt6:508yl0chzzy0000029 Sprint 2DDRP-2021 A 4 Story Points 2.0