[PIPE2D-1008] Confirm units of pfsArm flux are electrons/pixel and pfsMerged electrons/nm Created: 23/Mar/22  Updated: 20/May/23  Resolved: 20/May/23

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: arnaud.lefur
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File blue-arm gain-corrected.png     PNG File pfsArm_extraction_compared.png     PNG File pfsArms_vs_pfsMerged.png     PNG File pixel_per_nm.png     PNG File ratio between blue extraction.png     PNG File ratio between red extraction.png     PNG File red-arm gain-corrected.png     PNG File spectral_resolution_621.png    
Issue Links:
Relates
relates to PIPE2D-1024 Convert pfsMerged fluxes to electrons/nm Done
relates to PIPE2D-799 Correct for Jacobian when resampling Done
relates to DAMD-113 Write pfsArm in electrons Done
Story Points: 4
Sprint: 2DDRP-2022 C, 2DDRP-2022 D, 2DDRP-2022 E, 2DDRP-2022 F

 Description   

The expected units for the pfsArm is electrons per pixel and pfsMerged files are electrons per nanometre.

Please go step by step through the 2D DRP code and products, from the raw exposures to the pfsArm and pfsMerged files to confirm that the units of the latter two products are as expected.

Please document the details of each step in this ticket.

This is important in general , and to help determine the cause of the b/r flux ratio discrepancy.



 Comments   
Comment by price [ 23/Mar/22 ]

Units will not be electrons per nm with the code as it currently is.
Converting ADU to electrons requires specifying the parameter isr.doApplyGains=True.
The code does not convert to flux density, due to the desire to that the extracted spectra have units matching that of the image.

Note that "The expected units for the pfsArm and pfsMerged files are electrons per nanometre." is not specified in datamodel.txt. Rather, the flux units of pfsArm explicitly called out to be "counts".

Comment by rhl [ 23/Mar/22 ]

This is about the overall throughput, not the b/r discrepancy.

Paul's comments are correct, but the comparison is still valid. We should be able to interpret the numbers in the pfsArm/pfsMerged files in terms of electrons/nm (and how to do so should be in the datamodel).

Comment by arnaud.lefur [ 30/Mar/22 ]

So I spend a bit of time looking into this.
starting from the raw image to the pfsMerged data.
I've used my pseudo ISR, pseudo extraction code to compare with.
The plots are for visit 72034 which should be moon with offset(d_ra=1 deg)
I've selected a subset of 10 fibers from SUNSS_DIFFUSE.

on the red arm:
spectra :
ratio of extraction:
the ratio is close enough to 1, so I think I understand well enough what's being done.

on the blue arm:
spectra:
ratio of extraction :
here I'm being penalized by the a bad detectorMap on the blue end, which is useful to know, but not really the point here.

Anyhow, from what I'm seeing, I confirm Paul statement, the pfsArm fluxes are in counts (or electrons if doApplyGains).

Regarding, pfsMerged :

I end up with the same numbers if I divide the flux by the spectral resolution (nm per pixel) which gives me flux in electrons/nm

This balance slightly the flux between the two arms, but the absolute value is now ~12 times higher.
To get back on my feet, I just need to correct so that the totalFlux of the mergedArm stay constant.

Comment by rhl [ 31/Mar/22 ]

Looks good, but I'm still slightly confused.

You're plotting the pfsArm files against wavelength, but not applying a Jacobian, so are you really plotting per row, but labelling the rows with the corresponding wavelength?

What happened with 477 in the blue? A bad detector map everywhere? Which rerun/calib are you using for this work?

In the pfsMerged files, the units should be electrons/nm which is easy to think about away from the dichroics. Is this what you observe? What is the factor of c. 12 coming from? pixels to nm?

Comment by arnaud.lefur [ 31/Mar/22 ]

You're plotting the pfsArm files against wavelength, but not applying a Jacobian, so are you really plotting per row, but labelling the rows with the corresponding wavelength?
Right exactly, for the pfsArm, the x-axis can be either range(4176) (row number) or the matching wavelength for each row.

What happened with 477 in the blue? A bad detector map everywhere? Which rerun/calib are you using for this work?
Do you mean fiber 6 ? I checked a bit closely and I don't understand what's happening with the measured flux, the detectorMap is actually pretty good for that fiber pretty much all the way. If you compare fiber 6 and 296 which appear to have a very similar throughput :

you can see that the dotted line are pretty much consistent, from the pipeline, 296 is bad on the blue end which I blame the detectorMap for but 6 is bad at the red end and the detectorMap is not that bad there.
Also we can see small bands where the flux drops to 0, might be fiberProfiles ?
I've used hassan's: CALIB-SuNSS-20220322-dev , rerun: alefur/pipe2d-1008-new

In the pfsMerged files, the units should be electrons/nm which is easy to think about away from the dichroics. Is this what you observe? What is the factor of c. 12 coming from? pixels to nm?

right, I just divided the pfsArm flux (e-/row) by the spectral resolution (nm/row) (or multiplied by this curve )
that should be e-/nm strictly speaking so I did not apply any normalization after that.

Comment by rhl [ 31/Mar/22 ]

We should get to the bottom of fibre 6 (sorry, I mis-read the legend). It could be a problem in the fiberTrace or the normalisation or ...

The dropouts matter too. This is not unrelated to PIPE2D-1015 – there should be a bit set wherever we don't accurately extract the flux.

I still don't understand the pfsMerged case. The fluxes should be in e/nm and your values should agree with the pfsMerged everywhere except where the dichroic matters. So the blue and red side should agree, and not have any structure with lambda or arm. price: what am I missing?

Generated at Sat Feb 10 16:01:22 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.