[PIPE2D-547] Investigate and solve the problem of negative values in LSF Created: 10/Apr/20  Updated: 05/Jan/21  Resolved: 14/Apr/20

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: ncaplar Assignee: ncaplar
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Interpolation_effect.png     PNG File Interpolation_effect_with_April1520_version.png     PNG File LSF_Neven_fiber339_wavelength_800_in_2d.png     PNG File LSF_Neven_fiber339_wavelength_800.png     PNG File where_the_data_is.png    
Issue Links:
Relates
relates to PIPE2D-347 Provide a method that provides a PSF ... Done
relates to PIPE2D-333 Construct a 1D LSF from a 2D PSF Done
Story Points: 4
Sprint: 2DDRP-2021 A
Reviewers: hassan

 Description   

Investigate the problem of negative values reported by price in pfspipe.ipmu.jp/jira/browse/PIPE2D-333



 Comments   
Comment by ncaplar [ 10/Apr/20 ]

This is my priority for the week starting April 12

Comment by ncaplar [ 14/Apr/20 ]

I have evaluated what LSF would I expect at the position that price mentioned (fiber=339, wavelength=800nm, i.e., at x=1765 and y=2021). I get the result shown in the Figures (LSF_Neven_fiber339_wavelength_800.png and LSF_Neven_fiber339_wavelength_800_in_2d.png). I see no negative values in either 1d or 2d. 

The way forward:

1. I will investigate if I ever see any negative values on my own computer.
2. If I never see any negative values, I have to investigate further from where did these negative values appear. 

Comment by ncaplar [ 14/Apr/20 ]

I never see any negative values in any of my PSFs and LSFs at 1600 spots across the detector. I am concluding, based on this, that the negative values are happening ``somewhere'' downstream. I am investigating. 

Comment by ncaplar [ 14/Apr/20 ]

Paul correctly pointed out that I was investigating the different input data (I was using Sep12, and the default input data was Sep11). With this input set, I did find some positions that have negative values for their predictions.

Basically what happens is:

1. the algorithm selects two points on the same side (e.g., both below the wanted point or above the wanted point)

2. the algorithm does linear interpolation, and if these two inputs are nearby and the wanted point is far away, a small gradient can produce large enough difference that the predicted value drops below 0.

This is illustrated in Figure ``Interpolation_effect.png''.

The effect is more likely to happen in the areas of sparse data, as input data points are far away (Shown in ``where_the_data_is.png''). This is why we get negative values in the center where there are few input PSF spots.

The problem will go away with a denser sampling of input points, as part of PIPE2D-521, but the creation of that datasets is probably at least a week away.

Comment by ncaplar [ 14/Apr/20 ]

This was discussed on the technical telecon on Apr 13/Apr 14. The conclusions:

In the short term:

  1. Finish PIPE2D-521
  2.  Modify the Interpolation algorithm that is always positive definite

In the longer term:

     1. Improve the interpolation algorithm, and see how to get best results

Comment by rhl [ 14/Apr/20 ]

I'm not at all surprised that interpolation leads to negative values, it was certainly the explanation that came to mind. I assume that most are very close to zero.

This problem should go away once we interpolate the pupil illumination/Zernikes

Comment by ncaplar [ 17/Apr/20 ]

I have added Figure ``Interpolation_effect_with_April1520_version.png'' showing that at the same location that we had negative values in the previous version, LSF is well behaved using the newest version of the spots. I still have to investigate if this is the case across the full detector plane.

Generated at Sat Feb 10 15:54:44 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.