https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1290 DRP 2-D Pipeline <p>The metrics to evaluate the extraction accuracy should be defined to make a QA script.</p> PIPE2D-1290

Define metrics to evaluate the extraction accuracy

Task Normal Open Unresolved Unassigned Satoshi Hamano EDR QualityAssurance Tue, 29 Aug 2023 03:46:15 +0000 Fri, 13 Oct 2023 19:32:35 +0000 0 5 <p>Yabe-san's notebook produces the extraction residual image (calexp - image simulated from traces and pfsArm) and the residual/calexp image.</p> <p> </p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16304/16304_98042+r1_subtracted.pdf" title="98042 r1_subtracted.pdf attached to PIPE2D-1290">98042 r1_subtracted.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16305/16305_98042+r1_subtracted-divided.pdf" title="98042 r1_subtracted-divided.pdf attached to PIPE2D-1290">98042 r1_subtracted-divided.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p>Also, the stats (mean, median, and std dev) per 7x7 pix cell are plotted for the residual image.</p> <p> </p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16306/16306_98042+r1_mean.pdf" title="98042 r1_mean.pdf attached to PIPE2D-1290">98042 r1_mean.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16307/16307_98042+r1_median.pdf" title="98042 r1_median.pdf attached to PIPE2D-1290">98042 r1_median.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16308/16308_98042+r1_std.pdf" title="98042 r1_std.pdf attached to PIPE2D-1290">98042 r1_std.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p>I'd like to add a function to calculate the rms of the extraction residual for each fiber and for the whole detector.</p> <p>Residual/error image. np.sqrt(calexp.variance.array) is used as the error.</p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16337/16337_97821+b1_chi-2.pdf" title="97821 b1_chi-2.pdf attached to PIPE2D-1290">97821 b1_chi-2.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p>The chi^2 and the calexp average of each fiber are plotted. The fibers of bright objects and its adjacent fibers tend to have large chi^2. The example images of the residual/error, calexp, and mask for the fiber of the bright object is attached. The bumpy pattern is created by rearranging the distorted fiber trace into a straight line.</p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16339/16339_97821+b1_chistats-13.pdf" title="97821 b1_chistats-13.pdf attached to PIPE2D-1290">97821 b1_chistats-13.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p>   <span class="image-wrap" style=""><a id="16341_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/16341/16341_____________________________2023-09-07_15.59.09_720.png" title="____________________________2023-09-07_15.59.09_720.png" file-preview-type="image" file-preview-id="16341" file-preview-title="____________________________2023-09-07_15.59.09_720.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/16341/_thumb_16341.png" style="border: 0px solid black" /></a></span></p> <p> </p> <p>Thank you, Hamano-san! Can I make a few suggestions?</p> <ul> <li>In 97821..chi-2.pdf, could you change the color scheme so that positive residual are red and negative residuals are blue?</li> </ul> <ul> <li>In addition to chi^2, I would like to see chi because we are interested in the sign.</li> <li>Could we show sigma(chi) for each fiber? We can get sigma from the interquartile range, |Q3-Q1| /1.354, for each fiber. If things behave, we expect sigma~1. Deviations from unity is a bad sign.</li> </ul> <p>Thank you for your comments. I updated the figures.</p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16355/16355_97821+b1_chi_coolwarm.pdf" title="97821 b1_chi_coolwarm.pdf attached to PIPE2D-1290">97821 b1_chi_coolwarm.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16356/16356_97821+b1_chistats_v2.pdf" title="97821 b1_chistats_v2.pdf attached to PIPE2D-1290">97821 b1_chistats_v2.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p>The data points of different targetType are plotted with different colors.</p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/16359/16359_97821+b1_chistats_v3.pdf" title="97821 b1_chistats_v3.pdf attached to PIPE2D-1290">97821 b1_chistats_v3.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> Relates PIPE2D-1315 Development Rank 0|02qpjd:00r20060i200186403c19