https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-316 DRP 2-D Pipeline <p>Once <a href="https://pfspipe.ipmu.jp/jira/browse/SIM2D-93" title="Obtain Blue and NIR spot data" class="issue-link" data-issue-key="SIM2D-93"><del>SIM2D-93</del></a> has been implemented, the DRP pipeline should be updated if necessary to demonstrate the merging of arms.</p> <p>It is already understood that an initial implementation for arm merging is currently in place.</p> <p>The accuracy of the merging operation (and a method for determining the accuracy) need to be provided to this ticket. This is TBD.</p> PIPE2D-316

Demonstrate arm merging

Task Normal Done Done price hassan 2DDRP-V5 Fri, 25 Jan 2019 17:24:04 +0000 Fri, 13 Mar 2020 22:25:45 +0000 Wed, 13 Mar 2019 19:41:05 +0000 6.0 0 3 <p>With some hacking, I can build blue-side images, which demonstrates that the spot data (<a href="https://pfspipe.ipmu.jp/jira/browse/SIM2D-93" title="Obtain Blue and NIR spot data" class="issue-link" data-issue-key="SIM2D-93"><del>SIM2D-93</del></a>) works.<br/> To do this properly, we need:</p> <ul> <li>Transmission curves</li> <li>Verify blue detector size</li> <li>Blue bias frames</li> </ul> <p>After a bunch of mucking around, I managed to process both blue and red data through <tt>mergeArms.py</tt>. An example <tt>pfsMerged</tt> file is plotted above. There are a bunch of problems evident in the plot, but the main thing to notice is that the blue and red arms have been merged.</p> <p>I'll push some changes to ticket branches once I've cleaned them up.</p> <p>Currently the fiber assignments don't match between the blue and red images. I would therefore like to delay adding blue data to the integration test until we can get the simulator properly producing coordinated blue and red images (<a href="https://pfspipe.ipmu.jp/jira/browse/SIM2D-99" title="Build blue-arm data in simulator" class="issue-link" data-issue-key="SIM2D-99"><del>SIM2D-99</del></a> + <a href="https://pfspipe.ipmu.jp/jira/browse/SIM2D-94" title="Simulator should accept a PfsConfig instance" class="issue-link" data-issue-key="SIM2D-94"><del>SIM2D-94</del></a>).</p> <p><tt>merge.png</tt> shows that merging works, but the continuum is not continuous over the merge region (~650nm). I believe this is due to an incorrect normalisation in the flat field and/or fiber trace.</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/11818/11818_merge.png" style="border: 0px solid black" /></span></p> <p>How do we know what's right here?  Should we generate some fibres with bright objects with simple spectra (e.g. constant nu I_nu;  power laws; black bodies) to help assess our success here.</p> <p>That's what I'm doing right now.</p> <p><tt>coadd.png</tt>: two simulated exposures each with red+blue arms, run through the pipeline, and this is the resultant coadd spectrum (black) compared with the input simulator spectrum (constant F_nu @ 22 mag; red). The sky line subtraction is awful because it’s a braindead placeholder algorithm, but the continuum level looks about right. Looks like we've fixed the discontinuities!</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/11820/11820_coadd.png" style="border: 0px solid black" /></span></p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> from lsst.daf.persistence <span class="code-keyword">import</span> Butler butler = Butler(<span class="code-quote">"INTEGRATION/rerun/integration/pipeline"</span>) coadd = butler.get(<span class="code-quote">"pfsCoadd"</span>, catId=7, tract=0, patch=<span class="code-quote">"0,0"</span>, objId=3, numExp=4, expHash=0x1eab2e92) fig, axes = coadd.plot(plotSky=False, plotErrors=False, show=False) axes.axhline(3631e9*10**(-0.4*22.0), color=<span class="code-quote">"r"</span>) fig.show() </pre> </div></div> <p><tt>merged_fixed.png</tt>: demonstrates that the merged spectrum (<tt>pfsMerged</tt>: single exposure) is good too. The higher orange line is the flux standard (18 mag), while the lower lines are the science objects (22 mag).</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/11821/11821_merged_fixed.png" style="border: 0px solid black" /></span></p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> from lsst.daf.persistence <span class="code-keyword">import</span> Butler butler = Butler(<span class="code-quote">"INTEGRATION/rerun/integration/pipeline"</span>) merged = butler.get(<span class="code-quote">"pfsMerged"</span>, expId=32, arm=<span class="code-quote">"r"</span>) fig, axes = merged.plot() </pre> </div></div> <p>I'm finally happy with this. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a>, would you please take a look? This ticket was the home for simulator work as well, so there are a bunch of changes in several packages.</p> <p>Ok overall. Separate tickets can be raised later on points that could be missed.</p> <p>Merged to master.</p> Blocks PIPE2D-344 SIM2D-94 SIM2D-99 PIPE2D-335 SIM2D-93 SIM2D-104 Relates PIPE2D-523 PIPE2D-381 Development Epic Link PIPE2D-350 Rank 0|02qpt6:508yjj6 Reviewers hassan Sprint 2019 B 2DDRP-2019 C Story Points 1.0