[PIPE2D-1332] fitPfsFluxReference: Use 1D prior rather than 4D prior Created: 20/Nov/23  Updated: 22/Nov/23  Resolved: 22/Nov/23

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: sogo.mineo Assignee: sogo.mineo
Resolution: Done Votes: 0
Labels: flux-calibration
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Reviewers: price

 Description   

fitPfsFluxReference: Use 1D prior rather than 4D prior

One of bottlenecks of fitPfsFluxReference has turned out to be
scipy.interpolate.LinearNDInterpolator called many times by the objective function.
There, scipy.interpolate.LinearNDInterpolator interpolates 4D prior probability density function.

Because the prior PDF is made from broadband fluxes, the prior PDF is not very meaningful except
along the axis of effective temperature. If we marginalize the other three dimensions, we can do
without this slow interpolator.

In fact, we can improve speed by using a 1D prior. Here is a profile before the modification:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1    0.001    0.001 1958.247 1958.247 fitPfsFluxReference.py:302(runDataRef)
     1  105.904  105.904 1493.559 1493.559 fitPfsFluxReference.py:643(fitModelsToSpectra)
 15284  563.177    0.037 1386.914    0.091 fitPfsFluxReference.py:766(objective)

Here is a profile after the modification:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1    0.004    0.004 1659.727 1659.727 fitPfsFluxReference.py:302(runDataRef)
     1    0.016    0.016 1197.156 1197.156 fitPfsFluxReference.py:643(fitModelsToSpectra)
 18526    3.910    0.000 1191.519    0.064 fitPfsFluxReference.py:771(objective)

Though ncalls of objective gets larger after the modification, its percall gets 30% less,
which leads to fitModelsToSpectra running faster by 300 seconds / visit.

Quality of fitting does not change. We input 100 known spectra to get the following results:

                               4D   ( % stddev)    1D   ( % stddev)
----------------------------------------------------------------------
   RMS of errors of  'teff'  216.03 (   29.35 )  216.76 (   29.45 )
   RMS of errors of  'logg'    0.58 (   52.03 )    0.44 (   39.20 )
   RMS of errors of     'm'    0.99 (   62.27 )    0.86 (   53.76 )
   RMS of errors of 'alpha'    0.33 (   98.34 )    0.36 (  109.98 )
   RMS of errors of   'vel'   49.31 (  inf    )   49.31 (  inf    )
----------------------------------------------------------------------
  Mean of errors of  'teff' -148.32 (  -20.15 ) -144.36 (  -19.61 )
  Mean of errors of  'logg'    0.13 (   11.58 )    0.09 (    8.07 )
  Mean of errors of     'm'   -0.40 (  -24.86 )   -0.43 (  -26.87 )
  Mean of errors of 'alpha'    0.08 (   25.11 )    0.05 (   15.41 )
  Mean of errors of   'vel'   -1.75 ( -inf    )   -1.75 ( -inf    )

(For example, RMS of errors of 'teff' (in K) is sqrt of \frac{1}{n}\sum_{i} (teff^{out}_{i} - teff^{in}_{i})^2.
"% stddev" means this value over the standard deviation of {{teff^{in}_{i} }}.)



 Comments   
Comment by sogo.mineo [ 21/Nov/23 ]

Could you review this change?

Comment by sogo.mineo [ 22/Nov/23 ]

Merged. Thanks for reviewing.

Generated at Sat Feb 10 16:06:14 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.