https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-946 DRP 2-D Pipeline <p>The 1-D sky subtraction code doesn't seem to normalise the spectra correctly. I took one of the September PFI visits (68879) and labelled all the fibres in the third to tenth percentile of flux as <tt>SKY</tt>, then reduced the data.  The raw data (pfsMerged.flux with no SKY fibres) are  <span class="image-wrap" style=""><a id="14471_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/14471/14471_68879-r1-skysub.png" title="68879-r1-skysub.png" file-preview-type="image" file-preview-id="14471" file-preview-title="68879-r1-skysub.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/14473/_thumb_14473.png" style="border: 0px solid black" /></a></span>  and the sky models (pfsMerged.sky) are  <span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/14472/_thumb_14472.png" style="border: 0px solid black" /></span> .  The sky-subtracted spectra (also pfsMerged.flux) are  <span class="image-wrap" style=""><a id="14471_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/14471/14471_68879-r1-skysub.png" title="68879-r1-skysub.png" file-preview-type="image" file-preview-id="14471" file-preview-title="68879-r1-skysub.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/14471/_thumb_14471.png" style="border: 0px solid black" /></a></span>  and the problem is clearly seen.</p> PIPE2D-946

Sky subtraction doesn't take fibre sensitivity into account

Story Normal Done Done price rhl Fri, 12 Nov 2021 00:58:00 +0000 Fri, 3 Dec 2021 12:42:26 +0000 Tue, 30 Nov 2021 18:43:21 +0000 0 2 <p>I suspect that this comes from modelling the sky in normalised units (not raw counts) so that the sky estimates from different fibres can be combined, but not correctly transforming these normalised units back to per-fibre counts.</p> <p>I retrieved the normalisation values from the <tt>fiberProfiles</tt>, and found that the line intensity (bright, isolated line at 843.2491: 6-2 P1(3.5) close doublet) plotted against the normalisation has a large scatter. If the normalisation had been applied to the measured line intensities, this should be a horizontal line. If the normalisation had not been applied, this should be a sloped line. The normalisation comes from the flux in the extracted quartz flat, and it's strange that this should not be tightly correlated with the sky line flux. I wonder if this is related to engineering activities of some sort?</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/14476/14476_intensity_norm.png" style="border: 0px solid black" /></span></p> <p>Looking at the images (quartz.jpg and sky.jpg), the fibers appear to have different intensity patterns. Below is a horizontal cut at the position of a sky line for a quartz and sky. The variation seen here seems qualitatively consistent with what's seen in the plots. I conclude that problem is intrinsic to the data.</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/14480/14480_quartz-graph.jpg" style="border: 0px solid black" /></span><br/> <span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/14479/14479_sky-graph.jpg" style="border: 0px solid black" /></span></p> <p>I don't think any further work is possible without knowing exactly what the differences between the quartz (68345) and sky (68879) are.</p> <p>I made one commit in drp&#95;stella, to select sky fibers when none are provided.</p> <p>The sky (68879) is much more like a quartz taken after (69006).</p> <p><span class="image-wrap" style=""><img src="https://pfspipe.ipmu.jp/jira/secure/attachment/14497/14497_pipe2d-946.png" style="border: 0px solid black" /></span></p> <p>Paul's analysis looks right.</p> <p>We can merge the all-sky hack, but I think in general that it's better to write a modified `pfsConfig` file to a rerun that defines a set of sky fibres. Because the sky fibre assignment isn't part of the design it doesn't modify the pfsConfigId so the book-keeping all works.</p> <p>Merged the all-sky hack, as it gives us options for future cases.</p> Development Rank 0|zzs9ul:xy Reviewers rhl Sprint 2DDRP-2021 A11 2DDRP-2021 A12