If the background subtraction is good, then the most robust way to measure the second moments is probably to use a Gaussian-weighted second moment. In general one adapts the width of the weights to the data (via the well-known adaptive Gaussian moment of Bernstein and Jarvis 2002), but this may make the focus calculations a little trickier. Until we get the AGs into focus, I suspect that the adaptive approach will be needed.

We have code for the Bernstein and Jarvis algorithm available.

The "good background subtraction" is of course necessary for any measurement! So it's a precursor for this ticket. Looking at photometric errors relative to Gaia would probably be a good way to validate this part of the processing. This may not be practical until we have the AGs in focus.

Just in case... let me try to explain the background of this ticket... We are analyzing AG through-focus data to improve focus measurement to monitor the focus.

This is a plot comparing the second moment (and FWHM) vs flux for the stars in a half area of AG5 (w/o glass) made by Kawanomoto-san. The plot is for one exposure. As you see, second moment has dependency on flux (star brightness). The Hexapod z-position is 2.6 (AG best focus was z=2.8). He told that the trend is the same, regardless condition (camera, w/ or w/o glass, focus) So, we think FWHM is more robust than second moment, because the wing is cut.

I don't know how that FWHM is measured, but people generally convert it from a second moment – and it's a second moment that we need to e.g. focus. This plot, I think, shows that the second moments are being measured badly rather than that second moments are bad.

I think FWHM is measured by Gaussian-fit, (one of the standard output from Sextractor). 

Ah, OK. So it isn't a FWHM, is `2 sqrt(2 ln(2)) sqrt(sigma_xx + sigma_yy)` with the moments coming from a fit; that's very similar to the Jarvis and Bernstein approach and may well be good enough.

A closer look at the sextractor documentation shows that the moments calculated by sep are the isophotal second moments of pixels above the threshold https://sextractor.readthedocs.io/en/latest/Position.html

sep doesn't have the option of doing fits to the data, although the full sextractor appears to. 

A 2D Gaussian fit to the core of the PSF, based on the positions returned by sextractor, produces a much less variable value. 

The attached file above shows the sep returned second moments, and the fitted ones, as in the other plot, for one side of the AG image.

So the question is whether the "FWHM" is good enough. If it isn't, we will probably want to use the adaptive moments I referenced above.

I've got an implementation of the adaptive moments that's much more robust; I'm still a bit unsure about the best way to figure out the FWHM for the Gaussian kernel; it's supposed to be tuned to the FWHM size of the data, which we don't know in advance. 

The plots below show the estimated sizes for the original second moments, as a function of flux, adaptive moment with a gaussian kernel, and direct fit to a gaussian, for dZ in focus from -0.3 to +0.6.  The colours indicate the absolute value of dZ (purple = 0, red =0.6, in chromatic order), with diamonds for negative values, plusses for positive and circles for dZ = 0. 

The Bernstein and Jarvis code determines it iteratively, such that the measured alpha^2 is half the kernel alpha^2.

Here's some old C from SDSS adaptive.c

Thanks, that was very helpful. 

I've got the routine coded; I need to add in handling things like edge of image points and numeric overflow properly, and run it through a variety of data to test for robustness. 

The code has been implemented and tested in the simulation environment; I will run it on a large set of AG data to check for issues. 

I've run the code through a variety of inputs at different focal positions and found

• there is no dependence of the second moments on the flux of the sources
• the change in size through a focus sweep is clearly picked up
• when I look at the images for outlying values (particularly high or low sizes) they can generally be correlated with close pairs of sources (particularly as the spots get larger and physically overlap) or scattered light artificats.

The first figure below  shows flux vs size plots for the glass and no glass sides (colour coded by focus position; note that the focussed position is midway through the colour palette). There is more scatter for the faintest sources. That saturated sources have been filtered out, which explains the smaller number of points at the brightest fluxes; these are saturated in the focussed images but not in the more unfocussed.

The plot of spot size vs frameId shows the change in size through the focus sweep. The colour indicates the camera (red -> purple for 1 -> 6). The outliers in the green adn orange spots match with close pairs of sources.

The third plots is a similar plot for a sequence of images going back and forth between dZ = 0 and 0.57. Here the outliers can be attributed to close pairs (left red, right and left orange, right yellow) and scattered light artifacts (left blue and purple).

Thank you for great efforts and sharing the graphs. Could you tell which side is for region with and without glass?

They're oriented the same way they are in the camera images. 

I'm sorry... but "left and right" is sometimes confusing to me... you mean, left is with glass (intra-focus), and right is without glass (extra-focus)?

J Karr reports: done and tested on simulations, but won't be implemented until after melco tests

I see that INSTRM-1773 has been merged with master. The commit merged (aaba6edf) however is different from what I was testing in simulation (8c304523). There are some differences and possibly typos in python/agccActor/centroidTools.py.

After some minimal testings, it seems python/agccActor/centroidTools.py in commit 8c3045231e674453632ce493cfcd1e5855a53a96 (which was tested in simulation) is mostly correct. master doesn't run.

master runs in simulation if I revert python/agccActor/centroidTools.py to commit 8c3045231e674453632ce493cfcd1e5855a53a96's version.

For recording:
Yoshida-san ran the ticket branch (commit  49cb82b ) . The current branch worked and is good to merge to master.

Merged to master.