[RM-6126] Conduct new 1st pass candidate selection tests with the new finder

Created on 2020-11-13 17:05:35 by Mira Sarkis. % Done: 0

Following the tests conducted in #5625:

• Use at least Develop 0.16 and most probably v.18 once the following tasks have been integrated in the library. We list
  • pdf refactor #5815
  • deciding on the size reduction of secondpass window #6103
  • after allowing several candidates per window #6052
  • after toggling the library completely into hdf5 (especially if we are planning on using large datasets??)
• Run tests on more realistic datasets from both PFS and Euclid:
  • using different metrics to select extrema:
    • Pdf: pdf[extrema] > threshold
    • PdfDiff: pdf[extrema] > max(pdf) - threshold
    • PdfInt: (integrated Pdf arround extrema, may be too long to compute ?) PdfInt > threshold
    • prominence (~ local extrema significance): P[extrema] > threshold
    • normalized prominence: P[extrema]/max(P) > threshold
    • Merit (Merit = chisquare): merit[extrema] < threshold (nb may be spectrum dependent on the number of samples actually used in the fit, thus use merit/N)
    • MeritDiff: merit[extrema] < max(merit) + threshold (same rmq on number of samples)
    • MeritDtD: merit[extrema] < merit_ground - threshold, where merit_ground = DtD (ie noise weighted sum of squared spectrum samples, aka merit of a null model), same dependance on number of samples.
      Note that many information can be found in the output files, notably in linemodelsolve.linemodel_extrema.json (#ExtremaMerit) and linemodelsolve.linemodel.csv (#dTransposeD)
  • Ideally, we should try to do the minimum nb of runs (optimally, one run), while taking into account/update or deactivating @FilterOutNeighboringPeaksAndTruncate@
    one run means, compute the necessary stuff (basically Pdf/ prominence/merit), choose one of the criteria to choose the best 100 candidates and then do pos-processing analysis