https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1110 DRP 2-D Pipeline <p>Flux calibration reads broad-band information from <tt>pfsConfig</tt>. But the current flux calibration assumes <a href="https://github.com/Subaru-PFS/drp_stella/blob/f5cd80046ee3316d6ab732e30bb2226f629244c2/python/pfs/drp/stella/fitReference.py#L111" class="external-link" rel="nofollow">different names</a> from targetDB/gaiaDB, and therefore we need to unify the names to run flux calibration. We change <tt>filterName</tt> in flux calibration and use ones of targetDB and gaiaDB.<br/>  <br/> {{filterName}}s passed down from targeted and gaiaDB to pfsDesign and pfsConfig are:<br/> <span class="error">&#91;band&#93;</span>_ps1 for PS1<br/> <span class="error">&#91;band&#93;</span>_gaia for Gaia<br/>  <br/> Similarly, we can use {{filterName}}s of <span class="error">&#91;band&#93;</span>_hsc and <span class="error">&#91;band&#93;</span>_sdss for HSC and SDSS, respectively.</p> <p>EDIT</p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15447/15447_PIPE2D-1110-change-filter-names.py" title="PIPE2D-1110-change-filter-names.py attached to PIPE2D-1110">PIPE2D-1110-change-filter-names.py^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> has to be run in order not to break the integration tests and weekly tests after this branch is merged to the main branch.</p> PIPE2D-1110

Make match filterName between pfsConfig and flux calibration

Task Normal Done Done sogo.mineo Takuji Yamashita flux-calibration Mon, 7 Nov 2022 07:57:45 +0000 Mon, 19 Dec 2022 19:17:22 +0000 Fri, 9 Dec 2022 00:11:24 +0000 0 5 <p>In regard to HSC r2 and i2 filters: <br/> I propose to use r2/i2 filters instead of the old r/i filters for flux calibration unless users explicitly set the filterName of the old ones.</p> <p>The HSC filters are used to scale fluxes of science targets in the last part of flux calibration. Calculating flux calibration vectors does not use HSC photometries because FLUXSTD stars have PS1 photometries. Users do not distinguish which filters, r<img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/information.png" height="16" width="16" align="absmiddle" alt="" border="0"/> or r2(i2), are used in the HSC catalog. The new r2 and i2 filters have been in use since 2016. Therefore, I think that it is natural to use r2 and i2 filters for our flux calibration.</p> <p>For example, if i_hsc is set in filterName of pfsConfig, the flux calibration reads the filter transmission of i2.</p> <p>It might be useful to have an option in which the old i filter is read when i_old_hsc (or other nice names) is set. Does targetDB accept such names for the old filters, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=kiyoto.yabe" class="user-hover" rel="kiyoto.yabe">Kiyoto Yabe</a> ?</p> <p>We should adopt a naming scheme for the photometric bands that makes the filter curve explicit, without having to assume anything.</p> <p>Do you mean that we should use the names of i2 and i, which corresponds with the filter curve names? In that case, targetDB needs to use a name including i2 to designate the i2 filter although the HSC catalog uses a name without "2" even for i2 band. Or do we choose the i2 filter curve internally in the pipeline?</p> <p>Oh, the problem is the HSC SSP catalog doesn't distinguish between i1 and i2 (though other reductions may). Yeah, that's annoying. Maybe we need to map i_HSC to i2, i2_HSC to i2 and i1_HSC to i1?</p> <p>Yes. That is because images taken with i1 and i2 are coadded in DUD, and i1 magnitudes in WIDE are corrected to i2. A user can fetch i1 magnitudes from the old data releases, but the band name is i not i1.<br/>  </p> <blockquote><p>map i_HSC to i2, i2_HSC to i2 and i1_HSC to i1</p></blockquote> <p>I agree with this. I prefer i_old for i1. It is a bit more meaningful, but this is subjective.</p> <p>There is already a filter named "ENG-R1", so I don't think the name "r1" is good for the older "HSC-R". Yamashita-san and I think the filter names should be:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> HSC: {g,r_old,r2,i_old,i2,z,y}_hsc PS1: {g,r,i,z,y}_ps1 Gaia: {bp,rp,g}_gaia SDSS: {u,g,r,i,z}_sdss</pre> </div></div> <p>I am changing filter names in <tt>fitReference.py</tt> but I am afraid that the changes break the integration test (and weekly test). <tt>psfConfig</tt> in drp_stella_data must be modified. I'm not sure I am able to do it properly. </p> <p>Also, I believe that <tt>i_hsc</tt> and <tt>r_hsc</tt> should result in an error. It would be better than having to memorize whether i means HSC-I or HSC-I2.</p> <p>Once this branch is merged to the main branch, pfsConfig&#42;.fits and pfsDetection&#42;.fits have to be modified (<span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15447/15447_PIPE2D-1110-change-filter-names.py" title="PIPE2D-1110-change-filter-names.py attached to PIPE2D-1110">PIPE2D-1110-change-filter-names.py^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> can be used to modify the files).</p> <p>Could you review this PR?</p> <p>We need to coordinate merging this ticket with the necessary changes to the integration test and weekly. I suggest you update the integration test inputs (in drp&#95;stella&#95;data), run the integration test to confirm it works, and then I'll update the weekly after you merge.</p> <p>&gt; you update the integration test inputs (in drp_stella_data),<br/> I have already run the modified integration test. Should I push the change of drp_stella_data?</p> <p>Yes, please.</p> <p>I pushed <tt>tickets/<a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1110" title="Make match filterName between pfsConfig and flux calibration" class="issue-link" data-issue-key="PIPE2D-1110"><del>PIPE2D-1110</del></a></tt> to <tt>drp_stella_data</tt> and created a pull request for it. The integration test ran successfully.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a> Can I merge the branches of <tt>drp_stella</tt> and <tt>drp_stella_data</tt> to the main branch?</p> <p>Please go ahead. Thanks!</p> <p>I ran the script on <tt>/projects/HSC/PFS/weekly-20221201</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> (lsst-scipipe-3.0.0) pprice@tiger2-sumire:/projects/HSC/PFS/weekly-20221201 $ python ~/PIPE2D-1110-change-filter-names.py pfsConfig*.fits --goahead </pre> </div></div> <p>Merged. Thanks for the review and changing the weekly test.</p> <p>And now I've run the script on <tt>/projects/HSC/PFS/scienceSims/scienceSims-20221201</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> (lsst-scipipe-3.0.0) pprice@tiger2-sumire:/projects/HSC/PFS/scienceSims/scienceSims-20221201 $ python ~/PIPE2D-1110-change-filter-names.py pfsConfig*.fits --goahead </pre> </div></div> Relates OBSPROC-33 Development Rank 0|02qpjd:00r20060i209 Reviewers price