[PIPE2D-880] Slow execution of Zernike algorithm after PIPE2D-874 Created: 05/Aug/21  Updated: 03/Sep/21  Resolved: 03/Sep/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: ncaplar Assignee: ncaplar
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 4

 Description   

The execution time has slowed significantly after the implementation of PIPE2D-874. For an image in focus, the time on my machine has increased from below 2 seconds, to approx. 12 seconds. It is not immediately obvious if the same slowdown is happening on cluster as well. This is obviously somehow connected with applying offsets and downsampling introduced in PIPE2D-874, but during testing of the algorithm I did not notice such a stark slowdown. 



 Comments   
Comment by ncaplar [ 27/Aug/21 ]

The reason for large increase in the execution time is that Lancoz interpolation is 1. somewhat more time consuming and 2. my optimizing algorithm needs a large number of iterations to find center of light .

1. using galsim and Lanczos5 interpolation, a single iteration of the shifting oversampled image in focus and drawing it takes around 40 ms.
2. to find the shift that creates model image with the same center of light as the scientific image my algorithm was badly optimized and using 100 to 300 iterations. 

In this fashion, the algorithm was using many seconds (4+ seconds) to create an image in focus. 

Two improvements can be made:
1. For image in focus, it is actually trivial to find the position which makes center of light (as e.g., how it is done in the pipeline).
2. For images in defocus where I am centering image in order to reduce chi - further investigation is needed. (I might switch to using lsst.offset algorithm which is faster or reducing the number of iterations needed)

Comment by ncaplar [ 03/Sep/21 ]

I have implemented both improvements mentioned above (internal version of code 0.48a).

1. When studying images in focus and matching flux position, I am using straightforward adjustments, instead of running full optimizing algorithm.

2. Everything has been switched to lsst.offset algorithm. This now requires the code to run in lsst universe.

The implementation took much longer than expected. This is due to my mistakes and confusion with proper ordering of axis, and some fine details in how lsst.offset work. Additionally, when switching to lsst algorithm some dependencies are older version than I have been using (for instance I have been using pandas version 1.2 and pipeline has 0.24 available), so the code had to be adjusted to be compatible. 

Generated at Sat Feb 10 15:59:17 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.