https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-614 DRP 2-D Pipeline <p>Pupil illumination is the same at different levels of defocus, and only the wavefront changes. At the moment, I do iterative estimate which fits each spot at each defocus separately. This sometimes produces a consistent estimate of the illumination, but sometimes not. In particular, FRD estimate is often discrepant at different levels of defocus. Example.png shows an example of two spots. In the upper row, the estimate is inconsistent at two sides of defocus (frd_s values are very different). In the lower row, example where they are consistent. The estimation code should handle this properly, by analyzing images at the same time.</p> PIPE2D-614

Illumination estimate of the pupil should be constant across different defocuses for single spot

Task Normal Done Done ncaplar ncaplar Mon, 29 Jun 2020 12:45:13 +0000 Mon, 4 Jan 2021 20:23:19 +0000 Fri, 20 Nov 2020 13:59:23 +0000 0 1 <p>I have uploaded images showing the current state of the processing. All images are analyzed at once (although this can still be improved, to be discussed below). The code uses an algorithm from Tokovinin &amp; Heathcote 2006 to search for the best wavefront solution. This is a code I mentioned in <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-354" title="Estimate high-order Zernike from defocused images" class="issue-link" data-issue-key="PIPE2D-354"><del>PIPE2D-354</del></a> (and is the basis for <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-553" title="Estimate higher-order Zernike terms for large number of images" class="issue-link" data-issue-key="PIPE2D-553"><del>PIPE2D-553</del></a>, even though it is not mentioned explicitly there). </p> <p>The quality.png shows the chi*2 reduced of images as a function of defocus. It shows the maximal chi2 (i.e., effectively very close to just np.mean(sci_image2/var_image) ), and chi*2 after model is subtracted. I am also showing line showing improvement of log-factor of 2.3, which is roughly the level of improvement which I have been getting with single image fitting.</p> <p>I am then showing several images showing fitting at different refocused. One can see that I am missing some global dependence - observed how there is a systematic trend in the residuals (more red in the bottom left and top right, more blue in bottom right and top left.)</p> <p> </p> <p>I am then showing results in the focus, in 2d space and 1d space. Even though this is very high signal to noise spot so some deviations is expected, the results can certainly be improved.</p> <p> </p> <p>At the moment the code still effectively analyzes individual images, suggests solutions for each image, and then interpolates the solution before going to the next step in the iteration. The proper way would be to analyze all images at once. </p> <p>I have implemented the version of the code that is able to analyze multiple images at once in a relatively economic and sustainable fashion.</p> <p>The code works in several steps:</p> <p>1. Given the initial guess for the parameters, find the best wavefront that minimizes the likelihood of all images. To do that I move wavefront parameters one by one and recorded the change that this does to the images. I then solve the linear equation which aims to maximize the likelihood. This assumes that we are making small enough steps that the changes are linear and that we are close enough to the real solution that making these small steps makes sense. This is a relatively expensive step as creating the images for the change of each parameter takes time (around 30 to 40 seconds when working on 40 cores). </p> <p>2. There is an optimization algorithm that explores non-wavefront parameters. For probing of the non-wavefront parameters I also try to find the best wavefront parameters without recalculating the images, i.e., assuming that changes due to wavefront parameters changes are the same as calculated in step 1.    </p> <p>3. If you found the parameters which are better than at the beginning, go back to step one and update the full result. Iterate until you converged. </p> <p>Another improvement is the fact that I have discovered that 3 struts that hold the detector are not of the same width. In previous versions of the algorithm, they have been fixed to be the same, which was creating large problems for the algorithm. I am still trying to go to understand which numbers should be inserted, but a preliminary test in which the vertical struts is 33% larger than the other 2 struts gives a significant improvment.</p> <p>I have also added images showing the improvement results:</p> <p>1. Example_final_defocused - 2d results for one of the spots when defocused<br/> 2. Example_final_focused - 2d results for the same spot in focus<br/> 3. Example_1d_extraction_large - 1d result extraction<br/> 4. Example_1d_extraction - 1d result extraction, zoomed </p> <p>I will now close this ticket and use this algorithm to analyze defocused data from Subaru. </p> <p>The only improvement I hope to make is a more accurate description of struts, based on conversations with Jim and <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=fmadec" class="user-hover" rel="fmadec">fmadec</a>. I will open a separate ticket for this. </p> Blocks PIPE2D-631 Relates PIPE2D-654 PIPE2D-671 PIPE2D-670 PIPE2D-406 Development Rank 0|02qps2:j Sprint 2DDRP-2021 A Story Points 6.0