https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-81 DRP 2-D Pipeline <p>Need to be able to retrieve dithered flats, aperture definition flats (equivalent to dither=0), etc. One option is to load all flats from the Butler and sort them out in Python code; another is to define separate dataset types, or do something clever with DataIds. Please:</p> <ul class="alternate" type="square"> <li>Write down all the different types of flats you expect to be working with;</li> <li>Produce a straw-man design for how you want to organize them.</li> </ul> PIPE2D-81

Design access mechanism, dataset types for accessing flats from Butler

Sub-task PIPE2D-10 Major Done Done aritter swinbank Fri, 9 Sep 2016 17:45:17 +0000 Fri, 7 Oct 2016 05:12:26 +0000 Fri, 7 Oct 2016 05:12:26 +0000 0 3 <p>We will be working with dispersed Flats only, no photometric Flats. There is 2 kinds of dispersed Flats:</p> <ul> <li>no offset in x for aperture definition (identification of tracing of the FiberTrace apertures)</li> <li>dithered Flats for the construction of normalized Flats to take out the pixel-to-pixel sensitivity variations (~10 different offsets in x)<br/> We need a header keyword that tells us the shift in x: 0 for no shift +/- xShift. Currently this header keyword is "sim.slit.xoffset". We might want to change that to "slit.xoffset" or something similar.</li> </ul> <p>Flat calibration products are:</p> <ul> <li>a combined Flat with 0 xOffset for the aperture definition. I propose to call this calibration file "apDef" and add an apDef file to the data model and the mapper, following the standard file names and mapper entries for calibration products. According to the data model the file names should be<br/> "pfsApDef-%06d-%1d%1s.fits" % (visit0, spectrograph, arm)</li> <li>a normalized Flat which I propose to call "Flat" and stick to the definition in the data model and mapper.</li> </ul> <p>Tasks to create the calibration products:</p> <ul> <li>I propose to add a CalibTask "constructApDefTask" which searches for Flats with xOffset=0, runs the ISR, and combines multiple exposures to remove cosmic rays which could irritate the tracing procedure</li> <li>The normalized Flat is being constructed by the CalibTask "constructNormFlatTask", which already exists. Preferably multiple exposures of each xOffset should be combined to get rid of cosmic rays. Combination of multiple exposures with the same xOffset is not yet implemented.</li> </ul> <p>Why do we need your "apDef" files? We will be persisting the traces, not the files that go into their generation. See <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-47" title="make FiberTrace persistable" class="issue-link" data-issue-key="PIPE2D-47"><del>PIPE2D-47</del></a> and <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-68" title="Add FiberTrace to data model" class="issue-link" data-issue-key="PIPE2D-68"><del>PIPE2D-68</del></a> (which I have linked to this ticket). I think that this ticket needs to define how to get the butler to read and write the persisted traces, and that includes defining their format in the data model &#8211; it's in as a placeholder, </p> <blockquote><p>"pfsFiberTrace-%06d-%1d%1s.fits" % (visit0, spectrograph, arm)</p></blockquote> <p>but without any specification of the format beyond TBD.</p> <p>It's fine to keep the processed dithered flats as "Flat" in the data model. When you sort out the header keywords, provenance, etc. that'll need to be added to the data model description.</p> <p>Is there a ticket to handle multiple offsets with the same x, and CR removal etc.? If not, please file one.</p> <p>In the discussion with Craig we found that we will also have image Flats (no spectrograph), at least until the CCDs are permanently mounted in the spectrographs. These image Flats (short imFlats) will be used to include pixel-to-pixel sensitivity variations in the simulations, and to look for CCD defects. However, we both agreed that we won't need a new datamodel entry for these imFlats as they can easily be constructed by running the standard ISR and then median combine the postISR images.<br/> As Robert correctly points out we don't need apDef Files as we will persist the FiberTraceSets, so really the only Flat we need to persist is the normalized Flat. For the creation of the FiberTraceSets we will write a CalibTask that starts with the raw (dithered) Flats, finds the Flats with xOffset=0.0, runs the ISR, combines them, and traces the apertures.</p> <p>After discussion with RHL and Craig we decided to persist both image Flats and fiber Flats. The current 'Flat' will be renamed 'fiberFlat' to avoid confusion once and for all.</p> Relates PIPE2D-47 PIPE2D-68 Development Rank 0|ii00pz: Reviewers rhl Sprint 2014-17