https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1122 DRP 2-D Pipeline <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=shamano" class="user-hover" rel="shamano">Satoshi Hamano</a> reports that the detectorMap adjustment of 80906r1 has very poor quality.<br/> Running with <tt>w.2022.46</tt>, I get the following summary stats:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> reduceExposure.adjustDetectorMap INFO: Final result: chi2=107354.909901 dof=44438 xRMS=0.063492 yRMS=0.234515 xSoften=0.040791 ySoften=0.049777 from 34832 lines reduceExposure.adjustDetectorMap INFO: Stats <span class="code-keyword">for</span> OH: chi2=60810.024394 dof=19260 xRMS=0.081180 yRMS=0.234515 xSoften=0.000000 ySoften=0.345114 from 9630 lines reduceExposure.adjustDetectorMap INFO: Stats <span class="code-keyword">for</span> OI: chi2=0.000000 dof=0 xRMS=nan yRMS=nan xSoften=0.000000 ySoften=0.000000 from 0 lines reduceExposure.adjustDetectorMap INFO: Stats <span class="code-keyword">for</span> Trace: chi2=46544.885507 dof=25202 xRMS=0.060551 yRMS=nan xSoften=0.051768 ySoften=0.000000 from 25202 lines </pre> </div></div> <p>The "Final result" line looks like it might be acceptable at first glance, judging from the <tt>xSoften</tt> and <tt>ySoften</tt> values being ~ 0.05; though the yRMS value of 0.23 is concerning, one might think that it's just reflecting measurement errors. But when looking at the stats for OH lines, we see <tt>ySoften=0.345</tt> and that's just unacceptable.</p> <p>The expected position of lines using the adjusted detectorMap do not visually coincide with the lines on the image.</p> PIPE2D-1122

Investigate poor detectorMap adjustment in r1

Story Normal Done Done price price Thu, 1 Dec 2022 15:49:54 +0000 Thu, 2 Feb 2023 19:30:25 +0000 Thu, 8 Dec 2022 20:24:37 +0000 0 2 <p>I've become dissatisfied with the current sky linelist, and believe it's the main reason we're having trouble adjusting the detectorMap in this case. The main reason is that it merges the Osterbrock and Rousselot lists in an unprincipled manner. The Osterbrock list is linked to observations, while the Rousselot list is a theoretical list without line strengths (the original paper was focussed on NIR observations, and the linelist in the optical appears to have gotten tagged on for use by ESO). Merging the two lists without regard to the line identifications or strengths means that the wavelength of the merged lines can be inaccurate. Furthermore, by merging lines indiscriminately, it becomes difficult to identify lines that are blended.<br/> I'd like to replace the current sky linelist with a new one that I have constructed solely from the Osterbrock list (at least for the wavelength range it covers). I've only merged lines when they are within 0.1 nm and I believe there is reason to expect them to have the same strength (e/f and P1/P2, etc.).<br/> With the current linelist, running on 80906r1, I get (after a variety of fixes):</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> reduceExposure.adjustDetectorMap INFO: Final result: chi2=2483923.321630 dof=38826 xRMS=0.055762 yRMS=0.643215 xSoften=0.000000 ySoften=0.510687 from 19425 lines </pre> </div></div> <p>Notice that the ySoften value of ~0.5 pixels is telling us that there are severe systematic problems with the fit in the spectral dimension (the fit in the spatial dimension is excellent). This is running with an exclusion radius of 3 pixels, so blends should be ignored (but they aren't). The new linelist gives:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> reduceExposure.adjustDetectorMap INFO: Final result: chi2=25275.377851 dof=23700 xRMS=0.057707 yRMS=0.062246 xSoften=0.000000 ySoften=0.000000 from 11862 lines </pre> </div></div> <p>There are fewer lines (11k vs 19k) used in the fit because we have much more reliable blend identification, and that makes the fit much better: <tt>ySoften=0</tt>, which means that any errors in the fit are caused by the statistical noise in the data rather than systematics.</p> <p>Attached the notebook that I used to merge the Osterbrock lines and measure the (merged) line intensities. It also contains code to do a cross-correlation of the image and linelist for various detectorMap offsets, which might be helpful in the future.</p> <p>I think all the suggested changes and revised linelist look fine to me.</p> <p>Taking this back after review to work on additional detectorMap adjustment updates.</p> <p>Made a few additional changes based on the DRP technical meeting of 2022-12-05 to protect some lines from the exclusion zone.</p> <p>Ready for new review.</p> <p>Additional changes look fine. Might need to re-phrase one commit comment for drp_stella. See my comments in the pull request. </p> <p>Reworded that commit, and merged.</p> <p>Origin of the <tt>oh.dat</tt> file with the Osterbrock lines: <a href="https://cdsarc.cds.unistra.fr/viz-bin/cat/III/211" class="external-link" rel="nofollow">https://cdsarc.cds.unistra.fr/viz-bin/cat/III/211</a></p> Development Rank 0|02qpjd:00r20060i2001 Reviewers hassan