[PIPE2D-946] Sky subtraction doesn't take fibre sensitivity into account Created: 12/Nov/21  Updated: 03/Dec/21  Resolved: 01/Dec/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: rhl Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File 68879-r1-raw.png     PNG File 68879-r1-sky.png     PNG File 68879-r1-skysub.png     PNG File intensity_norm.png     PNG File intensity.png     PNG File pipe2d-946.png     JPEG File quartz-graph.jpg     PNG File quartz.png     JPEG File sky-graph.jpg     PNG File sky.png    
Sprint: 2DDRP-2021 A11, 2DDRP-2021 A12
Reviewers: rhl

 Description   

The 1-D sky subtraction code doesn't seem to normalise the spectra correctly. I took one of the September PFI visits (68879) and labelled all the fibres in the third to tenth percentile of flux as SKY, then reduced the data.  The raw data (pfsMerged.flux with no SKY fibres) are   and the sky models (pfsMerged.sky) are  .  The sky-subtracted spectra (also pfsMerged.flux) are   and the problem is clearly seen.



 Comments   
Comment by rhl [ 12/Nov/21 ]

I suspect that this comes from modelling the sky in normalised units (not raw counts) so that the sky estimates from different fibres can be combined, but not correctly transforming these normalised units back to per-fibre counts.

Comment by price [ 16/Nov/21 ]

I retrieved the normalisation values from the fiberProfiles, and found that the line intensity (bright, isolated line at 843.2491: 6-2 P1(3.5) close doublet) plotted against the normalisation has a large scatter. If the normalisation had been applied to the measured line intensities, this should be a horizontal line. If the normalisation had not been applied, this should be a sloped line. The normalisation comes from the flux in the extracted quartz flat, and it's strange that this should not be tightly correlated with the sky line flux. I wonder if this is related to engineering activities of some sort?

Comment by price [ 16/Nov/21 ]

Looking at the images (quartz.jpg and sky.jpg), the fibers appear to have different intensity patterns. Below is a horizontal cut at the position of a sky line for a quartz and sky. The variation seen here seems qualitatively consistent with what's seen in the plots. I conclude that problem is intrinsic to the data.


Comment by price [ 17/Nov/21 ]

I don't think any further work is possible without knowing exactly what the differences between the quartz (68345) and sky (68879) are.

I made one commit in drp_stella, to select sky fibers when none are provided.

Comment by price [ 30/Nov/21 ]

The sky (68879) is much more like a quartz taken after (69006).

Comment by rhl [ 01/Dec/21 ]

Paul's analysis looks right.

We can merge the all-sky hack, but I think in general that it's better to write a modified `pfsConfig` file to a rerun that defines a set of sky fibres. Because the sky fibre assignment isn't part of the design it doesn't modify the pfsConfigId so the book-keeping all works.

Comment by price [ 01/Dec/21 ]

Merged the all-sky hack, as it gives us options for future cases.

Generated at Sat Feb 10 16:00:14 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.