https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/DAMD-92 Data Model <p>The 1D pipeline computes the probability of a source to be a galaxy, a star or a QSO. The current data model provides redshift candidates for the type with the highest probability.</p> <p> </p> <p>This story is related to a major update of 1D pipeline DM. The goal is to include to the DM all the candidates whatever the probability of a source to be a galaxy, star or qso.</p> <p> </p> <p>We propose the following DM:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>HDU#0 PDU HDU#1 GALAXY_CANDIDATES Binary table [FITS BINARY TABLE] NGALAXY_CANDIDATES Colums for : CRANK INT32 Rank of galaxy candidate (best=0) Z FLOAT Redshift Z_ERR FLOAT Redshift error Z_PROBA FLOAT XXXX (dz=+/-3.e-3) SUBCLASS STR Subclassifications CFILE STR Continuum template file name LFILE STR Linecatalog ratio template file name MODELFLUX FLOAT[NROW] Spectrum model (unit: nJy) HDU#2 GALAXY_PDF Binary table [FITS BINARY TABLE] NPIX Colums for : PDF FLOAT[NROW] PDF marginalised over all models HDU#3 GALAXY_LINES Binary table [FITS BINARY TABLE] NLINE Columns for : LINENAME STR Line name LINEWAVE FLOAT Catalog wavelength for this line in vacuum (unit: nm) LINEZ FLOAT Redshift LINEZ_ERR FLOAT Redshift error LINESIGMA FLOAT Gaussian width (unit: nm) LINESIGMA_ERR FLOAT Error in gaussian width LINEVEL FLOAT Gaussian width (unit: km/sec) LINEVEL_ERR FLOAT Error in gaussian width LINEFLUX FLOAT Area in gaussian fit (unit: erg/cm^2/s) LINEFLUX_ERR FLOAT Flux error LINEEW FLOAT Equivalent width (unit: nm) LINEEW_ERR FLOAT Equivalent width error LINECONTLEVEL FLOAT Continuum level at line center (unit: nJy) LINECONTLEVEL_ERR FLOAT Error in continuum level at line center HDU#4 QSO_CANDIDATES Binary table [FITS BINARY TABLE] NQSO_CANDIDATES Colums for : CRANK INT32 Rank of QSO candidate (best=0) Z FLOAT Redshift Z_ERR FLOAT Redshift error Z_PROBA FLOAT XXXX (dz=+/-3.e-3) SUBCLASS STR Subclassifications MODELFLUX FLOAT[NROW] Spectrum model (unit: nJy) HDU#5 QSO_PDF Binary table [FITS BINARY TABLE] NPIX Colums for : PDF FLOAT PDF marginalised over all models HDU#6 QSO_LINES Binary table [FITS BINARY TABLE] NLINE Columns for : LINENAME STR Line name LINEWAVE FLOAT Catalog wavelength for this line in vacuum (unit: nm) LINEZ FLOAT Redshift LINEZ_ERR FLOAT Redshift error LINESIGMA FLOAT Gaussian width (unit: nm) LINESIGMA_ERR FLOAT Error in gaussian width LINEVEL FLOAT Gaussian width (unit: km/sec) LINEVEL_ERR FLOAT Error in gaussian width LINEFLUX FLOAT Area in gaussian fit (unit: erg/cm^2/s) LINEFLUX_ERR FLOAT Flux error LINEEW FLOAT Equivalent width (unit: nm) LINEEW_ERR FLOAT Equivalent width error LINECONTLEVEL FLOAT Continuum level at line center (unit: nJy) LINECONTLEVEL_ERR FLOAT Error in continuum level at line center HDU#7 STAR_CANDIDATES Binary table [FITS BINARY TABLE] NSTAR_CANDIDATES Columns for : CRANK INT32 Rank of star candidate (best=0) V FLOAT Star velocity (unit: kms-1) V_ERR FLOAT Star velocity error T_PROBA FLOAT XXXX SUBCLASS STR Subclassifications TFILE STR Template file name MODELFLUX FLOAT[NROW] Spectrum model (unit: nJy) HDU#8 STAR_PDF Binary table [FITS BINARY TABLE] NPIX Colums for : PDF FLOAT PDF related to best model</pre> </div></div> DAMD-92

Update 1D pipeline DM with multi-type candidates

Story Normal Done Done Pierre-Yves CHABAUD Pierre-Yves CHABAUD Tue, 27 Oct 2020 10:41:37 +0000 Fri, 28 Jan 2022 17:16:21 +0000 Tue, 11 Jan 2022 16:11:27 +0000 0 8 <p>The header of the FITS file should contains the version of the pipeline used to process spectrum</p> <p>How are you planning to track all the configuration parameters?</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=rhl" class="user-hover" rel="rhl">rhl</a>  It's a really good question. There might be lot of paramaters and not so easy to represent. The current parameter file is a JSON structured format whereas fits header is a flat format.</p> <p> </p> <p>An other solution could be to track only parameters which are different from default parameters. Indeed, the value of default parameters are embed in the pipeline version. But this solution implies a dynamic header format (keyword might appear or not). I'm not sure if it's such a good idea.</p> <p> </p> <p>How do you manage that on 2D side ?</p> <p>I agree to have a PDF for each type.  In addition, science users will need P(galaxy), P(QSO), and P(star).  Is that information included in the primary header?</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=msyktnk" class="user-hover" rel="msyktnk">Masayuki Tanaka</a> You're absolutly right. I update the PHDU with the CLASS keyword (should be GALAXY, QSO or STAR) and the computed values of P(galaxy), P(QSO) and P(star)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>HDU#0 PHDU CLASS KEYWORD Spectro classification: GALAXY, QSO, STAR P_GALAXY KEYWORD Probability to be a galaxy P_QSO KEYWORD Probability to be a QSO P_STAR KEYWORD Probability to be a star</pre> </div></div> <p>In accordance with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=msyktnk" class="user-hover" rel="msyktnk">Masayuki Tanaka</a> and <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a> proposition, I update the PHDU with pipeline and product versions. I also move classification results to a dedicated classification HDU</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>HDU#0 PHDU D1D_VER KEYWORD Version of the DRP_1D library D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline DAMD_VER KEYWORD Version of the data model PAR_FILE KEYWORD Parameters file name HDU#1 CLASSIFICATION CLASS KEYWORD Spectro classification: GALAXY, QSO, STAR P_GALAXY KEYWORD Probability to be a galaxy P_QSO KEYWORD Probability to be a QSO P_STAR KEYWORD Probability to be a star </pre> </div></div> <p>Regarding line measurement, wich unit should we use for the flux ? erg/cm^2/s or 10^-35 W/m2/Hz (more consistent with nJy)?</p> <p>Why not just use nJy?</p> <p>nJy is the unit for the spectrum flux density. As for the line flux, we have to integrate over the line profile, leading to a flux in W/m2 (or any other multiple like the vintage erg/cm2/s or nJ.Hz which I never saw but we could be pioneers on that one <img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/wink.png" height="16" width="16" align="absmiddle" alt="" border="0"/> ).</p> <p>Right; I was being stupid. So you meant to propose 10^-35 W/m2 (not "/Hz")? </p> <p>well the original mistake was mine actually <img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/tongue.png" height="16" width="16" align="absmiddle" alt="" border="0"/> so yes I meant W/m2 ...</p> <p>Description of "PDF" HDU will be change <font color="#1d1c1d">header from </font><tt>TTYPE2 = 'PDF'</tt><font color="#1d1c1d">  to </font><tt>TTYPE2 = 'ln PDF'</tt> to avoid future confusions.</p> <p>Current proposal for the datamodel (from <a href="https://github.com/Subaru-PFS/datamodel/blob/tickets/DAMD-92/datamodel.txt" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/datamodel/blob/tickets/DAMD-92/datamodel.txt</a>) :</p> <p>HDU#0 PHDU<br/> D1D_VER KEYWORD Version of the DRP_1D library<br/> D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline<br/> DAMD_VER KEYWORD Version of the data model<br/> PAR_FILE KEYWORD Parameters file name<br/> ZWARNING INT64 Quality flag</p> <p>HDU#1 CLASSIFICATION<br/> CLASS KEYWORD Spectro classification: GALAXY, QSO, STAR<br/> P_GALAXY KEYWORD Probability to be a galaxy<br/> P_QSO KEYWORD Probability to be a QSO<br/> P_STAR KEYWORD Probability to be a star</p> <p>HDU#1 GALAXY_CANDIDATES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NGALAXY_CANDIDATES<br/> Colums for :<br/> CRANK INT32 Rank of galaxy candidate (best=0)<br/> Z FLOAT Redshift<br/> Z_ERR FLOAT Redshift error<br/> Z_PROBA FLOAT Areak of the PDF peak (dz=+/-3.e-3)<br/> SUBCLASS STR Subclassifications<br/> CFILE STR Continuum template file name<br/> LFILE STR Linecatalog ratio template file name<br/> MODELFLUX FLOAT<span class="error">&#91;NROW&#93;</span> Spectrum model (unit: nJy)</p> <p>HDU#2 GALAXY_PDF Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NPIX<br/> Colums for :<br/> REDSHIFT FLOAT<span class="error">&#91;NPIX&#93;</span> Redshift grid<br/> PDF FLOAT<span class="error">&#91;NPIX&#93;</span> PDF marginalised over all models</p> <p>HDU#3 GALAXY_LINES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NLINE<br/> Columns for :<br/> LINENAME STR Line name<br/> LINEWAVE FLOAT Catalog wavelength for this line in vacuum (unit: nm)<br/> LINEZ FLOAT Redshift<br/> LINEZ_ERR FLOAT Redshift error<br/> LINESIGMA FLOAT Gaussian width (unit: nm)<br/> LINESIGMA_ERR FLOAT Error in gaussian width<br/> LINEVEL FLOAT Gaussian width (unit: km/sec)<br/> LINEVEL_ERR FLOAT Error in gaussian width<br/> LINEFLUX FLOAT Area in gaussian fit (unit: 10^-35 W/m2)<br/> LINEFLUX_ERR FLOAT Flux error<br/> LINEEW FLOAT Equivalent width (unit: nm)<br/> LINEEW_ERR FLOAT Equivalent width error<br/> LINECONTLEVEL FLOAT Continuum level at line center (unit: nJy)<br/> LINECONTLEVEL_ERR FLOAT Error in continuum level at line center</p> <p>HDU#4 QSO_CANDIDATES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NQSO_CANDIDATES<br/> Colums for :<br/> CRANK INT32 Rank of QSO candidate (best=0)<br/> Z FLOAT Redshift<br/> Z_ERR FLOAT Redshift error<br/> Z_PROBA FLOAT Areak of the PDF peak (dz=+/-3.e-3)<br/> SUBCLASS STR Subclassifications<br/> MODELFLUX FLOAT<span class="error">&#91;NROW&#93;</span> Spectrum model (unit: nJy)</p> <p>HDU#5 QSO_PDF Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NPIX<br/> Colums for :<br/> REDSHIFT FLOAT<span class="error">&#91;NPIX&#93;</span> Redshift grid<br/> PDF FLOAT<span class="error">&#91;NPIX&#93;</span> PDF marginalised over all models</p> <p>HDU#6 QSO_LINES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NLINE<br/> Columns for :<br/> LINENAME STR Line name<br/> LINEWAVE FLOAT Catalog wavelength for this line in vacuum (unit: nm)<br/> LINEZ FLOAT Redshift<br/> LINEZ_ERR FLOAT Redshift error<br/> LINESIGMA FLOAT Gaussian width (unit: nm)<br/> LINESIGMA_ERR FLOAT Error in gaussian width<br/> LINEVEL FLOAT Gaussian width (unit: km/sec)<br/> LINEVEL_ERR FLOAT Error in gaussian width<br/> LINEFLUX FLOAT Area in gaussian fit (unit: erg/cm^2/s)<br/> LINEFLUX_ERR FLOAT Flux error<br/> LINEEW FLOAT Equivalent width (unit: nm)<br/> LINEEW_ERR FLOAT Equivalent width error<br/> LINECONTLEVEL FLOAT Continuum level at line center (unit: nJy)<br/> LINECONTLEVEL_ERR FLOAT Error in continuum level at line center</p> <p>HDU#7 STAR_CANDIDATES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NSTAR_CANDIDATES<br/> Columns for :<br/> CRANK INT32 Rank of star candidate (best=0)<br/> V FLOAT Star velocity (unit: kms-1)<br/> V_ERR FLOAT Star velocity error<br/> T_PROBA FLOAT Area of the PDF peak<br/> SUBCLASS STR Subclassifications<br/> TFILE STR Template file name<br/> MODELFLUX FLOAT<span class="error">&#91;NROW&#93;</span> Spectrum model (unit: nJy)</p> <p>HDU#8 STAR_PDF Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NPIX<br/> Colums for :<br/> REDSHIFT FLOAT<span class="error">&#91;NPIX&#93;</span> Redshift grid<br/> PDF FLOAT<span class="error">&#91;NPIX&#93;</span> PDF related to best modelHDU#0 PHDU<br/> D1D_VER KEYWORD Version of the DRP_1D library<br/> D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline<br/> DAMD_VER KEYWORD Version of the data model<br/> PAR_FILE KEYWORD Parameters file name<br/> ZWARNING INT64 Quality flag</p> <p>HDU#1 CLASSIFICATION<br/> CLASS KEYWORD Spectro classification: GALAXY, QSO, STAR<br/> P_GALAXY KEYWORD Probability to be a galaxy<br/> P_QSO KEYWORD Probability to be a QSO<br/> P_STAR KEYWORD Probability to be a star</p> <p>HDU#1 GALAXY_CANDIDATES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NGALAXY_CANDIDATES<br/> Colums for :<br/> CRANK INT32 Rank of galaxy candidate (best=0)<br/> Z FLOAT Redshift<br/> Z_ERR FLOAT Redshift error<br/> Z_PROBA FLOAT Areak of the PDF peak (dz=+/-3.e-3)<br/> SUBCLASS STR Subclassifications<br/> CFILE STR Continuum template file name<br/> LFILE STR Linecatalog ratio template file name<br/> MODELFLUX FLOAT<span class="error">&#91;NROW&#93;</span> Spectrum model (unit: nJy)</p> <p>HDU#2 GALAXY_PDF Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NPIX<br/> Colums for :<br/> REDSHIFT FLOAT<span class="error">&#91;NPIX&#93;</span> Redshift grid<br/> PDF FLOAT<span class="error">&#91;NPIX&#93;</span> PDF marginalised over all models</p> <p>HDU#3 GALAXY_LINES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NLINE<br/> Columns for :<br/> LINENAME STR Line name<br/> LINEWAVE FLOAT Catalog wavelength for this line in vacuum (unit: nm)<br/> LINEZ FLOAT Redshift<br/> LINEZ_ERR FLOAT Redshift error<br/> LINESIGMA FLOAT Gaussian width (unit: nm)<br/> LINESIGMA_ERR FLOAT Error in gaussian width<br/> LINEVEL FLOAT Gaussian width (unit: km/sec)<br/> LINEVEL_ERR FLOAT Error in gaussian width<br/> LINEFLUX FLOAT Area in gaussian fit (unit: 10^-35 W/m2)<br/> LINEFLUX_ERR FLOAT Flux error<br/> LINEEW FLOAT Equivalent width (unit: nm)<br/> LINEEW_ERR FLOAT Equivalent width error<br/> LINECONTLEVEL FLOAT Continuum level at line center (unit: nJy)<br/> LINECONTLEVEL_ERR FLOAT Error in continuum level at line center</p> <p>HDU#4 QSO_CANDIDATES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NQSO_CANDIDATES<br/> Colums for :<br/> CRANK INT32 Rank of QSO candidate (best=0)<br/> Z FLOAT Redshift<br/> Z_ERR FLOAT Redshift error<br/> Z_PROBA FLOAT Areak of the PDF peak (dz=+/-3.e-3)<br/> SUBCLASS STR Subclassifications<br/> MODELFLUX FLOAT<span class="error">&#91;NROW&#93;</span> Spectrum model (unit: nJy)</p> <p>HDU#5 QSO_PDF Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NPIX<br/> Colums for :<br/> REDSHIFT FLOAT<span class="error">&#91;NPIX&#93;</span> Redshift grid<br/> PDF FLOAT<span class="error">&#91;NPIX&#93;</span> PDF marginalised over all models</p> <p>HDU#6 QSO_LINES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NLINE<br/> Columns for :<br/> LINENAME STR Line name<br/> LINEWAVE FLOAT Catalog wavelength for this line in vacuum (unit: nm)<br/> LINEZ FLOAT Redshift<br/> LINEZ_ERR FLOAT Redshift error<br/> LINESIGMA FLOAT Gaussian width (unit: nm)<br/> LINESIGMA_ERR FLOAT Error in gaussian width<br/> LINEVEL FLOAT Gaussian width (unit: km/sec)<br/> LINEVEL_ERR FLOAT Error in gaussian width<br/> LINEFLUX FLOAT Area in gaussian fit (unit: erg/cm^2/s)<br/> LINEFLUX_ERR FLOAT Flux error<br/> LINEEW FLOAT Equivalent width (unit: nm)<br/> LINEEW_ERR FLOAT Equivalent width error<br/> LINECONTLEVEL FLOAT Continuum level at line center (unit: nJy)<br/> LINECONTLEVEL_ERR FLOAT Error in continuum level at line center</p> <p>HDU#7 STAR_CANDIDATES Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NSTAR_CANDIDATES<br/> Columns for :<br/> CRANK INT32 Rank of star candidate (best=0)<br/> V FLOAT Star velocity (unit: kms-1)<br/> V_ERR FLOAT Star velocity error<br/> T_PROBA FLOAT Area of the PDF peak<br/> SUBCLASS STR Subclassifications<br/> TFILE STR Template file name<br/> MODELFLUX FLOAT<span class="error">&#91;NROW&#93;</span> Spectrum model (unit: nJy)</p> <p>HDU#8 STAR_PDF Binary table <span class="error">&#91;FITS BINARY TABLE&#93;</span> NPIX<br/> Colums for :<br/> REDSHIFT FLOAT<span class="error">&#91;NPIX&#93;</span> Redshift grid<br/> PDF FLOAT<span class="error">&#91;NPIX&#93;</span> PDF related to best model</p> <p>I am terribly sorry for being extremely slow.  Morishima-san at NAOJ (he is a database person) and I carefully looked at the datamodel as well as real psfZCandidates files from drp_1dpipe (ver.0.20.1) + drp_1d (ed36a29) + datamodel (w.2021.26).  The majority of the work is kindly done by Morishima-san and thanks go to him.</p> <p>HDU #0: We see TRACT, PATCH, CATID, OBJID, NVISIT, VHASH are gone.  They are in the file names, but a user may well rename the file.  So, it is easy to loose this key information.  We prefer to have them in the primary header.  I discussed with Ali offline, and I think it would be good to explicitly have the line list version in the header.  The keyword PAR_FILE may not be super informative and we could have the configs defined in PAR_FILE in the header if the configs are not too many.  Finally, this is not a datamodel issue, but where can I find the definition of Z_WARNING?<br/> HDU #1: In addition to probabilities, it would be good to have reduced chi-squares.  It is useful; we don't trust a fit with chi2_nu=1e9 even if P(galaxy)=1 and Z_PROBA=0.99.  Maybe such a bad fit is captured by Z_WARNING, but again, I was not able to find its definition.</p> <p>HDU #2: Where can I find the wavelengths of the best-fit model?  I guess you used the CRPIX, CRCAL, CD from pfsObject but it would be nice to copy it to pfsZCandidates.  Also, it might make sense to have the galaxy and QSO PDF in the same HDU so they can share the REDSHIFT column.  By the way, I do not understand the numbers in the REDSHIFT column (you can look at the sample files in datapack-0.20.1/validation.  This is not a datamodel issue, though.</p> <p>HDU #4: It would be nice to be explicit about the units in the header (LINEWAVE is nm, LINEVEL is km/s).   I see some of the Balmer lines are measured twice: once for absorption and the other for emission.  Is this intended?</p> <p>HDU #7: What does T_PROBA mean?  It is an integrated probability over what area?</p> <p>HDU #8: REDSHIFT does not mean a lot for stars.  Maybe we can use RADIAL_VELOCITY (or simply VELOCITY) instead?  The explanation of PDF should be 'PDF marginalized over all models', no?</p> <p>HDU #9: Do you plan to make line measurements for stars?  If so, it should be included here.</p> <p>First of all, you can find the correct definition of the datamodel <a href="https://github.com/Subaru-PFS/datamodel/blob/tickets/DAMD-92/datamodel.txt" class="external-link" rel="nofollow">here</a><br/> Some of it is still under development (<a href="https://pfspipe.ipmu.jp/jira/projects/PIPE1D/issues/PIPE1D-52" class="external-link" rel="nofollow">https://pfspipe.ipmu.jp/jira/projects/PIPE1D/issues/PIPE1D-52</a>)<br/> HDU #0 : All remarks have been taken into account. Z_WARNING will be defined in drp1d_pipe README if that's ok. For now it is restricted to 1 bit, telling if line measurement is deactivated, this definition is themporary. The major content of this flag is still under development, the specification (still not definitive, extracted from RM-5638) is :</p> <div class='table-wrap'> <table class='confluenceTable'><tbody> <tr> <td class='confluenceTd'>_. Bit Name </td> <td class='confluenceTd'>_. Binary Digit </td> <td class='confluenceTd'>_. Description </td> </tr> <tr> <td class='confluenceTd'> <b>INVALID_SPECTRA_FLUX</b> </td> <td class='confluenceTd'> 0 </td> <td class='confluenceTd'> Invalid spectra flux (Nans, Infs, ...) </td> </tr> <tr> <td class='confluenceTd'> <b>INVALID_NOISE</b> </td> <td class='confluenceTd'> 1 </td> <td class='confluenceTd'> Invalid noise (Nans, Infs, err=0, ...) </td> </tr> <tr> <td class='confluenceTd'> <b>SMALL_WAVELENGTH_RANGE</b> </td> <td class='confluenceTd'> 2 </td> <td class='confluenceTd'> Available wavelength range is too small </td> </tr> <tr> <td class='confluenceTd'> <b>NEGATIVE_CONTINUUM</b> </td> <td class='confluenceTd'> 3 </td> <td class='confluenceTd'> Negative continuum found (AMPLITUDE + 3 * AMPLITUDE_ERR &lt; 0) at all z <em>at all lambda? (VLB)</em> (=&gt; flat continuum pdf ?) </td> </tr> <tr> <td class='confluenceTd'> <b>BAD_CONTINUUMFIT</b> </td> <td class='confluenceTd'> 4 </td> <td class='confluenceTd'> Bad continum fit (=&gt; linemodel fit will be bad, except when polynomial added in the model) <em>how do we know? (VLB)</em> </td> </tr> <tr> <td class='confluenceTd'> <b>EXPECTED_LINES</b> </td> <td class='confluenceTd'> 5 </td> <td class='confluenceTd'> H_alpha / OII, ... lines detectability in case of truncated spectra _for one line, all? _ </td> </tr> <tr> <td class='confluenceTd'> <b>CANDIDATE_ELIMINATION</b> </td> <td class='confluenceTd'> 6 </td> <td class='confluenceTd'> Eliminate candidate at second-pass range border </td> </tr> <tr> <td class='confluenceTd'> <b>NULL_AMPLITUDES</b> </td> <td class='confluenceTd'> 7 </td> <td class='confluenceTd'> All amplitudes are null at all z (=&gt; flat pdf) </td> </tr> <tr> <td class='confluenceTd'> <b>PEAK_NOT_FOUND_PDF</b> </td> <td class='confluenceTd'> 8 </td> <td class='confluenceTd'> Pdf finder peaks has failed </td> </tr> <tr> <td class='confluenceTd'> <b>MAX_AT_BORDER_PDF</b> </td> <td class='confluenceTd'> 9 </td> <td class='confluenceTd'> Pdf extremum is too close the border </td> </tr> </tbody></table> </div> <p>HDU GALAXY_CANDIDATES: Indeed, what would be usefull is a goodness of fit estimation. The reduced chi-squares being one of the possibilities. We are curently investigating several criteria to do so. You are right we missed that column for each candidate (HDU #2. HDU #4 &amp; HDU #7), could be named "reliability" or "goodness_fit" ? </p> <p>HDU GALAXY_CANDIDATES model: We use the wavelength WavelengthArray from PfsObject. We can add CRPIX, CRCAL, CD from pfsObject to HDU #0 (as it is common to HDU #1 #4 and #7)</p> <p>HDU GALAXY_PDF: Galaxy and QSO pdf do not share the redshift column, as the solvers work on different redshift ranges. They are on a logarithmic scale</p> <p>HDU GALAXY_LINES: Yes this is intended. These lines can be either in emission or absorption, eventually simultaneously (with different velocity dispersion)</p> <p>HDU STAR_CANDIDATES: It means probability a posteriori of the star template. The sum of all templates prob being 1 =&gt; description to be changed.</p> <p>HDU STAR PDF: Name has been changed. Explanation is correct</p> <p>Line measurement for stars are not planned</p> <p>Thank you, Ali!  I am sorry for this delay, but the changes look good to me. If you expect the meaning of Z_WARNING to evolve with time, then it would be a good idea to have the definition in the fits header. If not, README is perhaps fine (but we could of course have them in the header even in this case, so the user does not have to go back to README). I do not have a strong opinion on reliability vs goodness_fit. You can pick one that best describes the parameter you choose to use.</p> <p>Any updates on this <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=ali.allaoui" class="user-hover" rel="ali.allaoui">Ali Allaoui</a>?</p> <p>branch tickets/<a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-92" title="Update 1D pipeline DM with multi-type candidates" class="issue-link" data-issue-key="DAMD-92"><del>DAMD-92</del></a> is rebased and ready to be merged</p> <p>ticket branch merged to master.</p> Relates PIPE1D-25 Development Rank 0|02qpq3:0a Sprint 2DDRP-2021 A 2DDRP-2021 A 6 2DDRP-2021 A 7 2DDRP-2021 A 8