https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-880 DRP 2-D Pipeline <p>The execution time has slowed significantly after the implementation of <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-874" title="Centering/offset/interpolation - fix to PIPE2D 778 and PIPE2D-777" class="issue-link" data-issue-key="PIPE2D-874"><del>PIPE2D-874</del></a>. For an image in focus, the time on my machine has increased from below 2 seconds, to approx. 12 seconds. It is not immediately obvious if the same slowdown is happening on cluster as well. This is obviously somehow connected with applying offsets and downsampling introduced in <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-874" title="Centering/offset/interpolation - fix to PIPE2D 778 and PIPE2D-777" class="issue-link" data-issue-key="PIPE2D-874"><del>PIPE2D-874</del></a>, but during testing of the algorithm I did not notice such a stark slowdown. </p> PIPE2D-880

Slow execution of Zernike algorithm after PIPE2D-874

Task Normal Done Done ncaplar ncaplar Thu, 5 Aug 2021 14:52:46 +0000 Thu, 2 Sep 2021 20:33:13 +0000 Thu, 2 Sep 2021 20:33:13 +0000 0 1 <p>The reason for large increase in the execution time is that Lancoz interpolation is 1. somewhat more time consuming and 2. my optimizing algorithm needs a large number of iterations to find center of light .</p> <p>1. using galsim and Lanczos5 interpolation, a single iteration of the shifting oversampled image in focus and drawing it takes around 40 ms.<br/> 2. to find the shift that creates model image with the same center of light as the scientific image my algorithm was badly optimized and using 100 to 300 iterations. </p> <p>In this fashion, the algorithm was using many seconds (4+ seconds) to create an image in focus. </p> <p>Two improvements can be made:<br/> 1. For image in focus, it is actually trivial to find the position which makes center of light (as e.g., how it is done in the pipeline).<br/> 2. For images in defocus where I am centering image in order to reduce chi - further investigation is needed. (I might switch to using lsst.offset algorithm which is faster or reducing the number of iterations needed)</p> <p>I have implemented both improvements mentioned above (internal version of code 0.48a).</p> <p>1. When studying images in focus and matching flux position, I am using straightforward adjustments, instead of running full optimizing algorithm.</p> <p>2. Everything has been switched to lsst.offset algorithm. This now requires the code to run in lsst universe.</p> <p>The implementation took much longer than expected. This is due to my mistakes and confusion with proper ordering of axis, and some fine details in how lsst.offset work. Additionally, when switching to lsst algorithm some dependencies are older version than I have been using (for instance I have been using pandas version 1.2 and pipeline has 0.24 available), so the code had to be adjusted to be compatible. </p> Development Rank 0|zzs8vk: Story Points 4.0