https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-1588 Instrument control development <p>In the past few days, I found the pfs_utils returns different result across different version, especially, 6.2.0 and 6.1.3. Currently, MCS actor uses pfs_utils 6.1.3 without any problem. However, pfs_utils 6.2.0 causes error in transformation when I pull data from database (see attached figure). Since the pfs_utils is now widely used in all different actors, consistency is important. Therefore, we should take care this issue before the engineering run </p> <p><span class="image-wrap" style=""><a id="15165_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15165/15165_Screen+Shot+2022-04-28+at+1.20.34+PM.png" title="Screen Shot 2022-04-28 at 1.20.34 PM.png" file-preview-type="image" file-preview-id="15165" file-preview-title="Screen Shot 2022-04-28 at 1.20.34 PM.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15163/_thumb_15163.png" style="border: 0px solid black" /></a></span> </p> INSTRM-1588

Inconsistency of coordinate transformation across different version of pfs_utils on different actors

Task Major Done Done rhl chyan EngRun Mon, 25 Apr 2022 08:49:45 +0000 Tue, 17 May 2022 05:14:08 +0000 Tue, 17 May 2022 05:14:08 +0000 0 5 <p>6.1.3 was from the middle of the 2021-11 run, 6.2.0 includes transform changes which were were running (and thus had tested) at the end of the run, plus a couple of tickets from after the run.</p> <p>If I am not being dim there is a bug in the new MeasureDistortion.clip() but have no idea whether it could show your problem. If I put a fix for that on a branch can you test?</p> <p>That bug fix did not fix Chi-Hung's problem. Leaving branch open, and leaving the .clip() change on the branch.</p> <p>So, I did two tests last night. Two phi-arm motor map runs. The result looks like the following figure with version 6.1.3. </p> <p><span class="image-wrap" style=""><a id="15184_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15184/15184_Figure+1-20.png" title="Figure 1-20.png" file-preview-type="image" file-preview-id="15184" file-preview-title="Figure 1-20.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15164/_thumb_15164.png" style="border: 0px solid black" /></a></span></p> <p>Those spots are exactly at expected positions. However, if I use version 6.2.0, they looks lie this. <br/> <span class="image-wrap" style=""><a id="15185_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15185/15185_Figure+1-21.png" title="Figure 1-21.png" file-preview-type="image" file-preview-id="15185" file-preview-title="Figure 1-21.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15165/_thumb_15165.png" style="border: 0px solid black" /></a></span><br/> The spots are everywhere!</p> <p>Chi-Hung will provide Hassan and Yuki input data to test the transform routines in <tt>pfs_utils</tt> and help to identify the problem.</p> <p>Chi-Hung to confirm that <tt>6.1.4</tt> and <tt>6.1.5</tt> work correctly.</p> <p>Uploading notebooks file for transformation <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15178/15178_INSTRM-1588.ipynb" title="INSTRM-1588.ipynb attached to INSTRM-1588">INSTRM-1588.ipynb^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> </p> <p>OK, now I have better picture. Please see the plot. This evening, I use MCS with different versions of pfs_utils to transform cobra locations from pixel to mm unit. Since I did not command cobra to move, so they are pretty close to the cobra center (red spot). As you can see, 6.1.2 and 6.1.3 return reliable transformations. </p> <p><span class="image-wrap" style=""><a id="15185_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15185/15185_Figure+1-21.png" title="Figure 1-21.png" file-preview-type="image" file-preview-id="15185" file-preview-title="Figure 1-21.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15184/_thumb_15184.png" style="border: 0px solid black" /></a></span> </p> <p>The plots above shows that tag 6.1.4 is not reliable, but this tag is not applied to any code on any branch in GitHub, so possibly a fork: <a href="https://github.com/Subaru-PFS/pfs_utils/commit/9d076ba123d0933baafd009bd0682944f740eaff" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/pfs_utils/commit/9d076ba123d0933baafd009bd0682944f740eaff</a> .</p> <p>If you have the opportunity, then a test with tag 6.1.5 would be great.</p> <p>(My apologies. I wrote an earlier comment following a ICS/PFI telecon discussion suggesting to check 6.1.4, but I did not check at that time that this tag is in fact invalid).</p> <p>6.1.4 is the code which was being used at the end of the Nov run. It was at the tip of the <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1462" title="Add clipping to geometry updated in PfiTransform" class="issue-link" data-issue-key="INSTRM-1462"><del>INSTRM-1462</del></a> branch, which shortly after got merged to master and tagged 6.1.5, but after a rebase on top of a branch which introduced a sky-pfi transform change (<a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1491" title="Fields moves on the PFI coordinates as time goes." class="issue-link" data-issue-key="INSTRM-1491"><del>INSTRM-1491</del></a>). </p> <p>6.1.4 was left at the end of that branch for posterity. We could add a more descriptive label name, or even a branch name if the GitHub web interface is confused.</p> <p>So checking against both 6.1.4 and 6.1.5 would be interesting. If 6.1.4 does not work I am worried, since it contained the last few RHL transform changes, some of which mattered quite a bit.</p> <p>According to Chi-Hung's latest plots, 6.1.4 is problematic, as well as 6.2.0 which seems to give different results to 6.1.4.</p> <p>Yuki and I independently find a bug in the sigma-clipping algorithm which was introduced after 6.1.3 . Preparing a fix. However I see Craig's comment above where this appears to be discounted. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a>: can you elaborate why your fix did not work?</p> <p>Fix pushed. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=chyan" class="user-hover" rel="chyan">chyan</a> could you test please?</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=chyan" class="user-hover" rel="chyan">chyan</a> ran with the change, but still saw the problem. So the fix is still on the branch, but it does not solve this problem.</p> <p>There was also a swap in two of the distort() args (scale and theta), which seemed far more plausible, but I never tracked down how that could have leaked out into mcs/fps/cobracharmer.</p> <p>The recent proposed fixes are on a git branch named "PIPE2D-1588". Given the time I will go ahead and rename that to "<a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1588" title="Inconsistency of coordinate transformation across different version of pfs_utils on different actors" class="issue-link" data-issue-key="INSTRM-1588"><del>INSTRM-1588</del></a>"</p> <p>Boring FYI: my fix on <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1588" title="Inconsistency of coordinate transformation across different version of pfs_utils on different actors" class="issue-link" data-issue-key="INSTRM-1588"><del>INSTRM-1588</del></a> was subsumed by Hassan's, so I removed my old branch and renamed Hassan's to <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1588" title="Inconsistency of coordinate transformation across different version of pfs_utils on different actors" class="issue-link" data-issue-key="INSTRM-1588"><del>INSTRM-1588</del></a>.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a> Please check the result of your fix. The transformation is still not good. </p> <p> <span class="image-wrap" style=""><a id="15187_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15187/15187_INSTRM-1588-d1-old.jpg" title="INSTRM-1588-d1-old.jpg" file-preview-type="image" file-preview-id="15187" file-preview-title="INSTRM-1588-d1-old.jpg"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15185/_thumb_15185.png" style="border: 0px solid black" /></a></span> </p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=chyan" class="user-hover" rel="chyan">chyan</a> Can you provide me with the frameId related to the MCS pixel coordinates you used to perform the transformation in the above plot please?</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=chyan" class="user-hover" rel="chyan">chyan</a> I think you should call pfa_utils by setting insr0t=0 anytime for 71M camera.</p> <p>By changing</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> pt = transformUtils.fromCameraName(<span class="code-quote">'rmod'</span>,altitude=90, insrot=teleInfo[<span class="code-quote">'insrot'</span>].values[0])</pre> </div></div> <p>to</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> pt = transformUtils.fromCameraName(<span class="code-quote">'rmod'</span>,altitude=90, insrot=0.)</pre> </div></div> <p>(and using the same parameters as before) I got the same result. (without this change, I see a few micron difference, which I thought acceptable yesterday...)</p> <p>Pushed the latest fix (previously in <tt>u/hassans/20220511a</tt>) to the ticket branch, such that it is ready to be merged to master.</p> <p>This fix re-introduces the basic algorithm in the <tt>MeasureDistortion._<em>call</em>_</tt> method that was used in the <tt>pfs_utils</tt> 6.1.3, ie., without sigma-clipping.</p> <p>I believe that the sigma-clipping algorithm is however still valid, but a little more work is required to check its robustness across a number of scenarios. Given that there is little time before the May Eng Run, this should be addressed in a separate ticket after that run.</p> <p>The change isn't in the sigma clipping, it's in the definition of the cost function (<tt>np.mean(d**n)</tt>; <tt>n = 2</tt> works; <tt>n = 1</tt> doesn't).</p> <p>With <tt>n=1</tt> the initial solution looks like this. <span class="image-wrap" style=""><a id="15188_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15188/15188_INSTRM-1588-d1-new.jpg" title="INSTRM-1588-d1-new.jpg" file-preview-type="image" file-preview-id="15188" file-preview-title="INSTRM-1588-d1-new.jpg"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15187/_thumb_15187.png" style="border: 0px solid black" /></a></span> . Note that the same number of ring fiducials are matched with both the initial and second match (but different fiducials). With <tt>n=2</tt> the two matches are to the same fiducials, and the final fit is good. In other words, while <tt>n=2</tt> works it's basically due to good luck.</p> <p>The solution is to add a prior on the rotation for the original ring fit; this seems to work well.</p> <p><span class="image-wrap" style=""><a id="15193_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15193/15193_Figure+1-32.png" title="Figure 1-32.png" file-preview-type="image" file-preview-id="15193" file-preview-title="Figure 1-32.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15188/_thumb_15188.png" style="border: 0px solid black" /></a></span> Here is the solution with initial cost function of <tt>np.sum(d) + alphaRot*theta**2</tt> with <tt>alphaRot = 1</tt> for the "ring" fit. Note also the change from <tt>np.mean</tt> to <tt>np.sum</tt>; this means that when we have lots of points the Likelihood should dominate the prior on theta.</p> <p>The code sets <tt>alphaRot=0</tt> for the final solution, so there's no bias due this prior (it'd be small anyway for any reasonable <tt>alphaRot</tt> as the number of points fitted is much larger).</p> <p>I have just pushed these changes to the ticket branch.  We need to make sure that <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=chyan" class="user-hover" rel="chyan">chyan</a> sees them and makes suitable changes in the calling code to take advantage of this work.</p> <p> </p> <p>One comment: if the algorithm (n=1) selected different fiducials in the first and second match, due to rotational symmetry. Is it good practice in any case of a prior, for Chi-Hung to select the original fiducials such that it is rotationally <em>asymmetric</em>? Eg pick 5 fiducials on the left of the PFI and 3 on the right?</p> <p>I have tested Robert's fix. Attached plot show that the current transformation.</p> <p><span class="image-wrap" style=""><a id="15193_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15193/15193_Figure+1-32.png" title="Figure 1-32.png" file-preview-type="image" file-preview-id="15193" file-preview-title="Figure 1-32.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15193/_thumb_15193.png" style="border: 0px solid black" /></a></span> </p> <p>The new fix consistent with the center positions of cobra. I would suggest this ticket to be closed and merge to mater.</p> <p>Merged RHL's fix to master.</p> Blocks INSTRM-1394 Development Rank 0|02qpja: Reviewers rhl Sprint PreEngRun05 F PreEngRun05 G