https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-703 DRP 2-D Pipeline <p>Following the 2D DRP tech telecon 2020-01-20, it was pointed out that the simulated data for F-star standards used by DRP do not cover the full wavelength range (they cut off at around 1.23 micron while the NIR limit is 1.26 micron). </p> <p>Please provide spectra for these standards over the full PFS range.</p> <p>For specific details of these F-star standards, contact Paul Price. The GA team may also need to be contacted.</p> PIPE2D-703

Provide F-star standards covering the full PFS spectral range

Story Normal Done Done Takuji Yamashita hassan Wed, 20 Jan 2021 19:09:36 +0000 Tue, 20 Jul 2021 18:03:16 +0000 Tue, 20 Jul 2021 18:03:16 +0000 0 6 <p>The <a href="https://github.com/Subaru-PFS/drp_instdata/tree/master/data/objects/fluxCal" class="external-link" rel="nofollow">spectra currently used for flux standards in the simulator</a> are from AMBRE, provided by <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=msyktnk" class="user-hover" rel="msyktnk">Masayuki Tanaka</a>.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=takuji.yamashita" class="user-hover" rel="takuji.yamashita">Takuji Yamashita</a>, could you provide the extrapolated version of <a href="https://github.com/Subaru-PFS/drp_instdata/tree/master/data/objects/fluxCal" class="external-link" rel="nofollow">these templates</a>?</p> <p> </p> <p>OK!</p> <p>Should I make a branch for this on /drp_instdata/data/objects/fluxCal/ and upload the extrapolated templates there?</p> <p>That would be great, thanks. Could you please upload only the spectra corresponding to the ones that already exist (we're trying to keep the size of the package from becoming overwhelmingly large) and update the <tt>README.txt</tt> file appropriately?</p> <p>I have placed the extrapolated spectra and an updated README.txt in <a href="https://github.com/Subaru-PFS/drp_instdata/tree/tickets/PIPE2D-703/data/objects/fluxCal" class="external-link" rel="nofollow">a new branch</a>. These spectra were generated in the ticket of <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-364" class="external-link" rel="nofollow">PIPE2D-364</a>. The data format was changed from the original version. Only flux density in nJy is stored, you need to use FITST headers for wavelength solution, and the pixel scale is 0.01 nm/pix now. If you have any questions, please feel free to contact me. </p> <p>I got a warning from <tt>git</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Encountered 6 file(s) that should have been pointers, but weren't: data/objects/fluxCal/p6500_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits data/objects/fluxCal/p6500_g+4.0_m0.0_t01_z-1.00_a+0.00.AMBRE_Extp.fits data/objects/fluxCal/p7000_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits data/objects/fluxCal/p7000_g+4.0_m0.0_t01_z-1.00_a+0.00.AMBRE_Extp.fits data/objects/fluxCal/p7500_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits data/objects/fluxCal/p7500_g+4.0_m0.0_t01_z-1.00_a+0.00.AMBRE_Extp.fits </pre> </div></div> <p>Could you please check that you're using <tt>git lfs</tt>?</p> <p>Adding files with a different format will require coordinated changes in <tt>drp_instmodel</tt>.</p> <p>The spectrum appears to be discontinuous at the red end: the continuum doesn't match.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> &gt;&gt;&gt; <span class="code-keyword">import</span> astropy.io.fits &gt;&gt;&gt; ff = astropy.io.fits.open(<span class="code-quote">"data/objects/fluxCal/p6500_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits"</span>) &gt;&gt;&gt; flux = ff[1].data.Flux &gt;&gt;&gt; <span class="code-keyword">import</span> numpy as np &gt;&gt;&gt; wavelength = np.arange(len(flux))*0.01 + 300 &gt;&gt;&gt; <span class="code-keyword">import</span> matplotlib.pyplot as plt &gt;&gt;&gt; plt.plot(wavelength, flux, <span class="code-quote">'k-'</span>) &gt;&gt;&gt; plt.show() </pre> </div></div> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/13340/13340_discontinuous.png" style="border: 0px solid black" /></span></p> <p>I did not use git lfs. I do again.</p> <p>According to <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-50" title="Data model of theoretical model spectra for flux calibration" class="issue-link" data-issue-key="DAMD-50">DAMD-50</a>, the discussion on the data format is not fixed yet. We need to decide that,  <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=msyktnk" class="user-hover" rel="msyktnk">Masayuki Tanaka</a> and <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a>?</p> <p> </p> <p>I made those extrapolation for the spectra which were convolved with LSF (Gaussian), while the stored spectra at 1200nm are ones before the LSF convolution. Please use them after the LSF convolution on your side. I can send the LSF-convolved spectra but I thought the unconvolved data is better because you can apply your LSF.</p> <p>The spectrum that was convolved by an LSF (a Gaussian with FWHM = 2.4 \AA) is overplotted. A small "jump" remains at 1200 nm. This jump is smaller than 1% which is our target accuracy of flux calibration. I believe this is not a serious problem for flux calibration, but it is nice if we can resolve this. So, please let me add this issue to my development item list after my first loop of development. </p> <p>A function fitting for extrapolation was done on a spectrum with small absorption lines and therefore the extrapolation function is an "average" including small absorption lines and is a bit lower than the continuum. This is why we find the jump there. </p> <p> </p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/13341/13341_convolution.png" style="border: 0px solid black" /></span></p> <p> </p> <p> </p> <p>The FITS files were uploaded to github again using git lfs. Please check it. </p> <p>The FITS files are still present in git, from the previous commit. This makes the git repo larger than it should be (this is the whole reason for <tt>git lfs</tt>). I'll fix it, as it requires editing history.</p> <p>I've fixed the commits on the branch. If you have any problems on your local branch, please <tt>git fetch &amp;&amp; git reset --hard origin/tickets/<a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-703" title="Provide F-star standards covering the full PFS spectral range" class="issue-link" data-issue-key="PIPE2D-703"><del>PIPE2D-703</del></a></tt> (be warned that this will delete any local changes).</p> <p>I've updated the simulator code that reads these files, and also the pipeline code that uses them.</p> <p>We'll also want to regenerate the weekly raw dataset before this merges. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a>, are you aware of any other coming changes that will require regenerating the weekly, so we can coordinate them?</p> <p>To my understanding <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a>, there are no other upcoming changes that require regenerating the weekly. <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-100" title="Add the design position angle to pfsDesign" class="issue-link" data-issue-key="DAMD-100"><del>DAMD-100</del></a>, <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-101" title="Save the outputs of shuffle" class="issue-link" data-issue-key="DAMD-101"><del>DAMD-101</del></a> and <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-105" title="Include which arms should be used in the pfsDesign/pfsConfig file" class="issue-link" data-issue-key="DAMD-105"><del>DAMD-105</del></a> do change <tt>PfsConfig</tt> and <tt>PfsDesign</tt> files, but as we agreed I will ensure that the previous versions of those files can be read, for a certain period (ie., when we schedule a new weekly raw dataset generation).</p> <p>When are we getting Sean's new object lists?  And when are we going to use different target lists for brn and bmn, that's a structural change to the weekly.  That'll require <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-105" title="Include which arms should be used in the pfsDesign/pfsConfig file" class="issue-link" data-issue-key="DAMD-105"><del>DAMD-105</del></a>, won't it?</p> <p>I've agreed with Paul offline about 30 mins ago that we will regenerate the weekly after <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-105" title="Include which arms should be used in the pfsDesign/pfsConfig file" class="issue-link" data-issue-key="DAMD-105"><del>DAMD-105</del></a>, <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-100" title="Add the design position angle to pfsDesign" class="issue-link" data-issue-key="DAMD-100"><del>DAMD-100</del></a>, <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-101" title="Save the outputs of shuffle" class="issue-link" data-issue-key="DAMD-101"><del>DAMD-101</del></a> are implemented in any case, given that I should be able to complete them in the next few days.</p> <p>I believe this is ready to merge, and we should include it in the new generation of sims.</p> <p>Paul has reviewed the changes.</p> <p>Merged.</p> Development Rank 0|zzs4wj:zzc Reviewers hassan Sprint 2DDRP-2021 A 2 2DDRP-2021 A 7 Story Points 2.0