https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/REDMINE1D-53 1D Redmine <p><em><font color="#505f79"> Created on 2020-09-22 12:35:53 by Didier Vibert. % Done: 100</font></em></p> <p>Il s'agit de comparer le linemodel complet en log-lambda: ie comparaison du linemodel sur un spectre régulier vs linemodel complet sur spectre ré-échantillonné en lambda après avoir validé (dans #5972) le fait que sur un spectre log-lambda templatefitting et templatefittinglog (calcul du moindre carré en fourrier ou non) donne le même résultat à la précision numérique près. Note: avant l'implémentation de #5989, seul le continu est fitté avec un spectre rebinné en log-lamda, le linemodel lui-même est fitté avec le spectre initial. Les comparaisons avec et sans loglambda, dans ce cadre, ont été effectuées dans le ticket: #5429-24 </p> <p>La différence issue de cette comparaison sera alors entièrement due au ré-échantillonnage du spectre. Discuter ensuite l'amélioration de celui-ci, en particulier via #5993 et le réglage du pas de ré-échantillonnage log en fonction de la LSF(lambda).</p> REDMINE1D-53

[RM-5994] loglambda: comparer la qualité du résultat avec échantillonnage régulier

Task Normal Done Done Redmine-Jira Migtation Redmine-Jira Migtation Fri, 4 Jun 2021 01:20:57 +0000 Wed, 5 Jul 2023 07:01:57 +0000 Wed, 5 Jul 2023 07:01:57 +0000 0 1 <p>Comment by Mira Sarkis on 2021-07-01 16:46:31:<br/> TODO notes:</p> <ul> <li>run linemodel on log-rebinned spectra (note that log-rebinned spectra are obtained by running TFLOG on regular spectra)</li> <li>compare redshift values and pdfs? as a metric</li> <li>conduct comparison on pfs spectra, notably pfs7</li> </ul> <p>Comment by Mira Sarkis on 2021-07-08 15:35:10:<br/> Launched PFS7 dataset with an additional option in param.json to rebin spectra and templates:</p> <ul> <li>output directory: /net/CESAM/amazed/msarkis/pfs7/output_7July_lm_logrebinnedSpc</li> </ul> <p>By comparing these results to those of develop_0.22.0 (/net/CESAM/amazed/validation_tests/release_0.22/output_pfs7_0.22.0_tplr_v10/):</p> <ul> <li>we lose 16 spectra, using sigma = 9.214E-5</li> <li>Mismatch increases by 11</li> <li>Spurious increases by 9</li> <li>Success decreases by 18</li> <li>les PDFs changent: pas mal de pdf s'affaissent et les candidats "secondaires" peuvent changer</li> </ul> <p>Comment by Vincent Le Brun on 2021-07-09 13:58:23:<br/> and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)</p> <p>Comment by Mira Sarkis on 2021-07-12 08:11:12:<br/> Vincent Le Brun wrote in #note-13:<br/> &gt; and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)</p> <p>With sigma=2e-4, and by comparing to results from develop_0.22.0: </p> <ul> <li>we lose 15 spectra</li> <li>spurious increases by 10</li> <li>mismatch increases by 5</li> </ul> <p>Edit:<br/> with same sigma, we have:</p> <ul> <li>11 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)</li> <li>26 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)</li> </ul> <p>Comment by Vincent Le Brun on 2021-07-12 15:59:44:<br/> Mira Sarkis wrote in #note-14:<br/> &gt; Vincent Le Brun wrote in #note-13:<br/> &gt; &gt; and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)<br/> &gt; <br/> &gt; With sigma=2e-4, and by comparing to results from develop_0.22.0: <br/> &gt; * we lose 15 spectra<br/> &gt; * spurious increases by 10<br/> &gt; * mismatch increases by 5<br/> &gt; <br/> &gt; Edit:<br/> &gt; with same sigma, we have:<br/> &gt; * 11 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)<br/> &gt; * 26 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)</p> <p>we should <b>gain</b> 15 spectra then (or 11 bad went good and 26 good went bad) ?</p> <p>Comment by Mira Sarkis on 2021-07-12 16:18:26:<br/> Vincent Le Brun wrote in #note-15:</p> <p>&gt; &gt; <b>Edit</b> :<br/> &gt; &gt; with same sigma, we have:<br/> &gt; &gt; * 11 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)<br/> &gt; &gt; * 26 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)<br/> &gt; <br/> &gt; we should <b>gain</b> 15 spectra then (or 11 bad went good and 26 good went bad) ?<br/> Unfortunately we lose 15 spectra and yes: 11 bad went good and 26 good went bad.</p> <p>Comment by Mira Sarkis on 2021-07-19 13:49:59:<br/> Time gain, per spectrum for:</p> <ul> <li>@precomputeContinuumFit@ is small, ie. 0.11 sec : (1.72 sec for lm+log vs 1.833 sec for develop)</li> <li>@firstpass@ is also small, ie. 0.242 sec : ( 27,935 sec for lm+log vs 28.177 sec for develop)</li> <li>@secondpass@ is of 1.864 sec per spectra : ( 37,5156 sec for lm+log vs 39,38sec for develop)</li> </ul> <p>Comment by Vincent Le Brun on 2021-09-02 14:37:34:<br/> Mira Sarkis wrote in #note-14:<br/> &gt; Vincent Le Brun wrote in #note-13:<br/> &gt; &gt; and if you take sigma = 2e-4 what is the evolution? (sigma=9e-5 gives a 3sigma threshold below 3e-4, which is quite strict...)<br/> &gt; <br/> &gt; With sigma=2e-4, and by comparing to results from develop_0.22.0: <br/> &gt; * we lose 15 spectra<br/> &gt; * spurious increases by 10<br/> &gt; * mismatch increases by 5<br/> &gt; <br/> &gt; Edit:<br/> &gt; with same sigma, we have:<br/> &gt; * 11 good results in develop_0.22.0, that become bad (e.g., 223966, 248087, 367183, 521921, 639189)<br/> &gt; * 26 bad results in develop_0.22.0, that become good (e.g., 386152(close peaks), 440293(interesting case), 481145, 645259, 729994)</p> <p>None of the lost objects listed here are good spectra (one of the OII lines is corrupted/absent). I consider this as acceptable loss</p> <p>Comment by Didier Vibert on 2021-09-08 14:48:24:<br/> ajouter un paramètre pour utiliser ou non l'échantillonnage log dans le linemodel (uniquement dans le cas de ré-échantillonnage d'un spectre non-log, car sinon il n'y a pas le choix...)</p> <p>Comment by Mira Sarkis on 2021-09-10 09:17:13:<br/> Merge requests:<br/> pylibamazed: <a href="https://gitlab.lam.fr/CPF/cpf-redshift/-/merge_requests/205" class="external-link" rel="nofollow">https://gitlab.lam.fr/CPF/cpf-redshift/-/merge_requests/205</a><br/> dataset-params: <a href="https://gitlab.lam.fr/amazed/dataset-parameters/-/merge_requests/1" class="external-link" rel="nofollow">https://gitlab.lam.fr/amazed/dataset-parameters/-/merge_requests/1</a></p> <p>Comment by Pierre-yves Chabaud on 2021-09-15 09:35:22:<br/> Merged on @develop@ : @e24503a@</p> Development Rank 0|zzs7io: