[INSTRM-1817] Implement code for fitting PSF templates to out of focus AG images Created: 07/Dec/22  Updated: 04/Jan/23  Resolved: 04/Jan/23

Status: Won't Fix
Project: Instrument control development
Component/s: ics_agccActor
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: karr Assignee: karr
Resolution: Won't Fix Votes: 0
Labels: EngRun
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File set1b_offsetsYAv.png     PNG File set1b_offsetsY.png     PNG File set1_offsetsXAv.png     PNG File set1_offsetsX.png     PNG File set1_offsetsYAv.png     PNG File set1_offsetsY.png    
Issue Links:
Blocks
is blocked by INSTRM-1821 Propogate template fitting informatio... Won't Fix
Sprint: PreEngRun09Dec

 Description   

Implementation of code for fitting model PSF templates for very out of focus AG images, based on positions in the focussed images. 



 Comments   
Comment by karr [ 07/Dec/22 ]

I have an implementation that works for the fitting; it takes a template, an image, and a position from the focussed image and uses cross correlation to determine the offset from the expected position. 

 

Comment by karr [ 07/Dec/22 ]

There is, of course, the larger question of what we do with this; the process of determining the seeing and picking the best template, and determining the in focus images will need to be done at a much higher level than the centroiding code. 

Comment by karr [ 07/Dec/22 ]

Also note that the routine will happily try to fit PSFs to noise in the cases where the images are blank due to shutter failure; how to filtering these out is not yet added.

Comment by karr [ 14/Dec/22 ]

I've updated the AgcCmd.py code to include a template method in addition to sep, to read default values of the templates (for dZ=0.58), and to read in templates for an explicit dZ value. The dZ and reference frame are passed via the cParm dictionary; however the mechanism for setting the reference frame is not currently determined.

Given those values, the template will retrieve reference positions from the database. 

 

Comment by karr [ 14/Dec/22 ]

Updated update; calling expose with ref=1 will set the reference positions and focus. Templates with dZ=0.58 will automatically be loaded; these can be explicitly updated if needed. 

 

Comment by yuki.moritani [ 14/Dec/22 ]

Thank you Jennifer, for modification. Could you explain a little more how to call expose command?

When we call agcc with dz~0.58, we call with ref=1? This option only accepts dz~+0.58?

Or, should we call agcc with ref=1 option to set reference z position? (I imagined this case; you set reference z position (m2pos3) and calculate offset of z position, and select proper template. )

Comment by hassan [ 15/Dec/22 ]

J Karr reports: I've pushed some code; Chi-Hung had a good suggestion for INSTRM-1821 which I have implemented. I need to do some testing on my end before asking Yoshida-san to do so.

Comment by karr [ 15/Dec/22 ]

When the actor is initialized, it will load the dz=0.58 templates as a default, set the reference focus to 0 and the spot positions to a zero length data frame. 

Calling the updateTemplate command with dz=0.60 would load the templates for  dz = 0.6; it will stay at this value until updated again. 

If you call expose with ref=1 it will store the returned spot information in the centroid parameters dictionary and update the reference focus value. After that, if you call expose with the centroid method set to template, it will use those values and do the template fitting. 

For the other parameters, if template fitting isn't used, we don't have the sextractor values. Instead, the adaptive second moments are used (set to 0 if non converged), the peak values are taken from the maximum value inside the fitting box. The 0 moment for the spot is currently set to zero; we'd need to implement some sort of aperture flux with a large aperture to do this. 

I could set things up to automatically load the appropriate template for each exposure after the reference values are set, but this would involve reloading the same file a lot. 

Also, note that the fitting will try to fit to noise when the spot is too faint to be detected in the out of focus images, or the shutter didn't open. 

 

Comment by karr [ 15/Dec/22 ]

I've tested the code that can be called outside the actor. The sequence for calling is

  • call expose with ref=1 and cMethod='sep' on a focussed image
  • move to the unfocussed position
  • if the dZ is not 0.57, update the template with for updateTemplate dZ=value
  • call expose with cMethod=template

If you call expose with cMethod=template, and reference data hasn't been set, it will return an empty set.

If you call expose with cMethod=template with the wrong reference data it will try to fit, but will do it badly, and the second moments will be nonsensical. 

 

Comment by karr [ 15/Dec/22 ]

Some updated plots for the template fitting

All plots use the sextractor results from a focussed image as a reference, with saturated or edge sources filtered out. The dZ value used is taken from the m2_pos3 position in the database, minus the reference frame position.

The colours indicate the camera (red > purple for 1> 6)

The top plot shows a focus sweep with the median Y offset for each camera image. The median is used to reject outliers (ie, close pairs, sources near edge, empty images due to shutter failure, and sources that are too faint to detect in unfocussed images. The horizontal line is at 0, the vertical at dZ = 0.57. The upper figure shows the results from the template fitting, the lower from sextractor. There is a steadily changing offset with focus position. The trend in the sextractor results is similar, but the scatter is much larger in unfocussed images.

The second plot shows the median X offset, which shows no trend.

The third and fourth plots are similar to the first, but showing all points, rather than just the median, to illustrate the scatter.

The fifth plot shows a closeup of the third plot, highlighting the focussed positions. The single point at 0 is the sextractor reference frame. The corresponding template positions show a small scatter around these values. The results for the two other focussed positions (at -0.025) show similar offsets for both methods.

I've visually inspected the returned positions for a set of data, compared to the fitted template, to make sure that the returned results make sense.

Comment by karr [ 15/Dec/22 ]

 

Comment by hassan [ 04/Jan/23 ]

It is planned to correct the AG focus for the upcoming Feb 2023 Run 10.

Generated at Sat Feb 10 16:39:56 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.