Comment by Mira Sarkis on 2021-07-01 16:46:31:
TODO notes:

• run linemodel on log-rebinned spectra (note that log-rebinned spectra are obtained by running TFLOG on regular spectra)
• compare redshift values and pdfs? as a metric
• conduct comparison on pfs spectra, notably pfs7

Comment by Mira Sarkis on 2021-07-08 15:35:10:
Launched PFS7 dataset with an additional option in param.json to rebin spectra and templates:

• output directory: /net/CESAM/amazed/msarkis/pfs7/output_7July_lm_logrebinnedSpc

By comparing these results to those of develop_0.22.0 (/net/CESAM/amazed/validation_tests/release_0.22/output_pfs7_0.22.0_tplr_v10/):

• we lose 16 spectra, using sigma = 9.214E-5
• Mismatch increases by 11
• Spurious increases by 9
• Success decreases by 18
• les PDFs changent: pas mal de pdf s'affaissent et les candidats "secondaires" peuvent changer

Comment by Vincent Le Brun on 2021-07-09 13:58:23:
and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)

Comment by Mira Sarkis on 2021-07-12 08:11:12:
Vincent Le Brun wrote in #note-13:
> and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)

With sigma=2e-4, and by comparing to results from develop_0.22.0:

• we lose 15 spectra
• spurious increases by 10
• mismatch increases by 5

Edit:
with same sigma, we have:

• 11 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)
• 26 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)

Comment by Vincent Le Brun on 2021-07-12 15:59:44:
Mira Sarkis wrote in #note-14:
> Vincent Le Brun wrote in #note-13:
> > and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)
>
> With sigma=2e-4, and by comparing to results from develop_0.22.0:
> * we lose 15 spectra
> * spurious increases by 10
> * mismatch increases by 5
>
> Edit:
> with same sigma, we have:
> * 11 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)
> * 26 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)

we should gain 15 spectra then (or 11 bad went good and 26 good went bad) ?

Comment by Mira Sarkis on 2021-07-12 16:18:26:
Vincent Le Brun wrote in #note-15:

> > Edit :
> > with same sigma, we have:
> > * 11 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)
> > * 26 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)
>
> we should gain 15 spectra then (or 11 bad went good and 26 good went bad) ?
Unfortunately we lose 15 spectra and yes: 11 bad went good and 26 good went bad.

Comment by Mira Sarkis on 2021-07-19 13:49:59:
Time gain, per spectrum for:

• @precomputeContinuumFit@ is small, ie. 0.11 sec : (1.72 sec for lm+log vs 1.833 sec for develop)
• @firstpass@ is also small, ie. 0.242 sec : ( 27,935 sec for lm+log vs 28.177 sec for develop)
• @secondpass@ is of 1.864 sec per spectra : ( 37,5156 sec for lm+log vs 39,38sec for develop)

Comment by Vincent Le Brun on 2021-09-02 14:37:34:
Mira Sarkis wrote in #note-14:
> Vincent Le Brun wrote in #note-13:
> > and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)
>
> With sigma=2e-4, and by comparing to results from develop_0.22.0:
> * we lose 15 spectra
> * spurious increases by 10
> * mismatch increases by 5
>
> Edit:
> with same sigma, we have:
> * 11 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)
> * 26 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)

None of the lost objects listed here are good spectra (one of the OII lines is corrupted/absent). I consider this as acceptable loss

Comment by Didier Vibert on 2021-09-08 14:48:24:
ajouter un paramètre pour utiliser ou non l'échantillonnage log dans le linemodel (uniquement dans le cas de ré-échantillonnage d'un spectre non-log, car sinon il n'y a pas le choix...)

Comment by Mira Sarkis on 2021-09-10 09:17:13:
Merge requests:
pylibamazed: https://gitlab.lam.fr/CPF/cpf-redshift/-/merge_requests/205
dataset-params: https://gitlab.lam.fr/amazed/dataset-parameters/-/merge_requests/1

Comment by Pierre-yves Chabaud on 2021-09-15 09:35:22:
Merged on @develop@ : @e24503a@