-
Type:
Task
-
Status: Done (View Workflow)
-
Priority:
Normal
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: None
-
Labels:
fitPfsFluxReference: Use 1D prior rather than 4D prior
One of bottlenecks of fitPfsFluxReference has turned out to be
scipy.interpolate.LinearNDInterpolator called many times by the objective function.
There, scipy.interpolate.LinearNDInterpolator interpolates 4D prior probability density function.
Because the prior PDF is made from broadband fluxes, the prior PDF is not very meaningful except
along the axis of effective temperature. If we marginalize the other three dimensions, we can do
without this slow interpolator.
In fact, we can improve speed by using a 1D prior. Here is a profile before the modification:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.001 0.001 1958.247 1958.247 fitPfsFluxReference.py:302(runDataRef)
1 105.904 105.904 1493.559 1493.559 fitPfsFluxReference.py:643(fitModelsToSpectra)
15284 563.177 0.037 1386.914 0.091 fitPfsFluxReference.py:766(objective)
Here is a profile after the modification:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.004 0.004 1659.727 1659.727 fitPfsFluxReference.py:302(runDataRef)
1 0.016 0.016 1197.156 1197.156 fitPfsFluxReference.py:643(fitModelsToSpectra)
18526 3.910 0.000 1191.519 0.064 fitPfsFluxReference.py:771(objective)
Though ncalls of objective gets larger after the modification, its percall gets 30% less,
which leads to fitModelsToSpectra running faster by 300 seconds / visit.
Quality of fitting does not change. We input 100 known spectra to get the following results:
4D ( % stddev) 1D ( % stddev)
----------------------------------------------------------------------
RMS of errors of 'teff' 216.03 ( 29.35 ) 216.76 ( 29.45 )
RMS of errors of 'logg' 0.58 ( 52.03 ) 0.44 ( 39.20 )
RMS of errors of 'm' 0.99 ( 62.27 ) 0.86 ( 53.76 )
RMS of errors of 'alpha' 0.33 ( 98.34 ) 0.36 ( 109.98 )
RMS of errors of 'vel' 49.31 ( inf ) 49.31 ( inf )
----------------------------------------------------------------------
Mean of errors of 'teff' -148.32 ( -20.15 ) -144.36 ( -19.61 )
Mean of errors of 'logg' 0.13 ( 11.58 ) 0.09 ( 8.07 )
Mean of errors of 'm' -0.40 ( -24.86 ) -0.43 ( -26.87 )
Mean of errors of 'alpha' 0.08 ( 25.11 ) 0.05 ( 15.41 )
Mean of errors of 'vel' -1.75 ( -inf ) -1.75 ( -inf )
(For example, RMS of errors of 'teff' (in K) is sqrt of \frac{1}{n}\sum_{i} (teff^{out}_{i} - teff^{in}_{i})^2.
"% stddev" means this value over the standard deviation of {{teff^{in}_{i} }}.)