The problem is presumably that there are multiple local minima, otherwise the initial simplex wouldn't matter. Are the differences significant? If so, we'll have to switch to something that explores the full parameter space (e.g. by simulated annealing), and/or use MCMC to evaluate the errors. Sounds slow!

Another option would be to take more cobra tracks through the spots; I think that'd be better

I have ran 101 iterations of the algorithm for November, January and February data.
The scatter of the solutions around the median solution for each spot is shown at (note that scaling is different for November panel!!!) !sigma_of_solutions.png|thumbnail! - for some reason the image is not rendering for me?

The quantity I am showing is a mean scatter, where I calculated scatter along x and y dimensions independently, and then take mean of those two values.

As we can see the scatter of solutions is large, and dependent on the position on the plane. For example, look at the distribution of solution for the spot in absolute top right corner for November run (left panel - arrow point to median solution, small black dots and solutions from individual runs). We see that the solution is very degenerate because of the lack of constraining power along one of the dimensions

Having said that, even in best case (bottom left), there is still quite large scatter (50 microns in each dimension)

Discussed on ICS telecons on March 18, 2022 and March 16, 2022. The algorithm has sufficient precision, given the quality of the data. Will be tested with data where there is actual double crossing of black dots across the focal plane.