[RM-6637] [linemeas] correct Line Flux variance estimation to include variance due to line width

Created on 2021-08-03 08:39:29 by Didier Vibert. % Done: 0

In the context of line measurement, in the case of Line Flux estimation using Gaussian fitting (as opposed to direct integration) we are using the following.

given the 2 fitted parameters, amplitude A, and line width w, the Flux F is computed as:
@F = sqrt(2 pi) * A * w@

then the variance of this flux estimator is currently computed as:
@var(F) = 2pi w^2 var(A)@,
var(A) coming from the analytical expression of the linear-least square estimator of the amplitude.

This formula is wrong since it does not include the fact that w is estimated and as such has it's own variance and covariance with A.

To correct for this we need to add the estimation of these two terms var(w) and covar(w,A) or an analytical approximation of them...
Since the fitting of w is done by least-square minimization by looping on a grid of velocity dispersion, we can estimate the width of the peak in pdf(w) to infer var(w)
For the covariance between w and A, we should analyze the 2D pdf(A,w) and fit the peak by a 2D Gaussian.

Or we could use the analytical expression derived in https://www.osapublishing.org/abstract.cfm?URI=ao-46-22-5374 (eq 19)
var(F) = 3 sqrt(pi) * sigma^2 * w /dx,
where

• sigma is the noise standard deviation at the line position (assuming it is uncorrelated pixel 2 pixel, and with constant variance)
• dx is the pixel size (same units than w, typically wavelength in Angstroms)

Note: when/if we will use a non-linear-least square minimizer (eg Levenberg-Marquardt), we could use the estimated returned covariance matrix to compute the Flux variance.