https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-997 DRP 2-D Pipeline <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a> noticed that the solutions of black dots optimizations are different between February and January run (even though the setup is the same). This is not unexpected - the black dot optimization algorithm has some inherit uncertainty because the initial simplex for Nelder-Mead is generated from random distribution. <br/> The goal is to estimate what is the distributions of the results between runs that operate on the same dataset. As Robert states in the comments, we need to estimate if these differences are significant. Depending on the result, it might be needed to modify/improve the algorithm. </p> PIPE2D-997

Measurement error in black dot optimization routine

Task Normal Done Done ncaplar ncaplar Fri, 4 Mar 2022 17:28:35 +0000 Fri, 18 Mar 2022 20:23:27 +0000 Fri, 18 Mar 2022 20:23:27 +0000 0 3 <p>The problem is presumably that there are multiple local minima, otherwise the initial simplex wouldn't matter. Are the differences significant? If so, we'll have to switch to something that explores the full parameter space (e.g. by simulated annealing), and/or use MCMC to evaluate the errors. Sounds slow!</p> <p>Another option would be to take more cobra tracks through the spots; I think that'd be better</p> <p>I have ran 101 iterations of the algorithm for November, January and February data. <br/> The scatter of the solutions around the median solution for each spot is shown at (note that scaling is different for November panel!!!) !sigma_of_solutions.png|thumbnail! - for some reason the image is not rendering for me?</p> <p>The quantity I am showing is a mean scatter, where I calculated scatter along x and y dimensions independently, and then take mean of those two values.</p> <p>As we can see the scatter of solutions is large, and dependent on the position on the plane. For example, look at the distribution of solution for the spot in absolute top right corner for November run (left panel - arrow point to median solution, small black dots and solutions from individual runs). We see that the solution is very degenerate because of the lack of constraining power along one of the dimensions <span class="image-wrap" style=""><a id="15126_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15126/15126_example_of_bot_left_corner_Nov_run.png" title="example_of_bot_left_corner_Nov_run.png" file-preview-type="image" file-preview-id="15126" file-preview-title="example_of_bot_left_corner_Nov_run.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15123/_thumb_15123.png" style="border: 0px solid black" /></a></span></p> <p>Having said that, even in best case (bottom left), there is still quite large scatter (50 microns in each dimension) <span class="image-wrap" style=""><a id="15126_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15126/15126_example_of_bot_left_corner_Nov_run.png" title="example_of_bot_left_corner_Nov_run.png" file-preview-type="image" file-preview-id="15126" file-preview-title="example_of_bot_left_corner_Nov_run.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15126/_thumb_15126.png" style="border: 0px solid black" /></a></span></p> <p>Discussed on ICS telecons on March 18, 2022 and March 16, 2022. The algorithm has sufficient precision, given the quality of the data. Will be tested with data where there is actual double crossing of black dots across the focal plane.</p> Relates PIPE2D-954 Development Rank 0|02qpq3:r3s00000000008w Sprint 2DDRP-2022 B Story Points 2.0