[PIPE2D-1024] Convert pfsMerged fluxes to electrons/nm Created: 15/Apr/22  Updated: 11/Jun/22  Resolved: 11/Jun/22

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Major
Reporter: price Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File pipe2d-1024.png    
Issue Links:
Relates
relates to DAMD-97 Add missing 2D pipeline outputs Done
relates to PIPE2D-1008 Confirm units of pfsArm flux are elec... Done
Story Points: 2
Sprint: 2DDRP-2022 D, 2DDRP-2022 E
Reviewers: hassan

 Description   

pfsMerged fluxes are currently in electrons per wavelength sample (since the wavelength sampling is similar to the original sampling, the fluxes are roughly in units of electrons per pixel like in the pfsArm files). We agreed in the Princeton meeting today that it would be better if the pfsMerged were in units of electrons/nm.

  • Change the units of the pfsMerged fluxes.
  • Check that this doesn't break the calibrated fluxes in the pfsObject files.
  • Document the flux units of pfsArm and pfsMerged in datamodel.txt.


 Comments   
Comment by hassan [ 03/Jun/22 ]

This blocks Kiyoto Yabe from his throughput analysis work.

Comment by price [ 11/Jun/22 ]

pfsMerged files now have flux units of electrons per nm, as demonstrated in this plot from visit=68351 (a quartz with PFI):

reduceExposure.py ~/pfs/Subaru --calib ~/pfs/Subaru/CALIB-PFI-20211220 --rerun pipe2d-1024 --id visit=68351 -j 2 -c isr.doFlat=False doAdjustDetectorMap=False doMeasureLines=False repair.doCosmicRay=False --no-versions
mergeArms.py ~/pfs/Subaru --calib ~/pfs/Subaru/CALIB-PFI-20211220 --rerun pipe2d-1024 --id visit=68351 -c doSubtractSky1d=False

import matplotlib.pyplot as plt
from lsst.daf.persistence import Butler
from pfs.drp.stella.interpolate import calculateDispersion
butler = Butler("/home/pfs/pfs/Subaru/rerun/pipe2d-1024")
red = butler.get("pfsArm", visit=68351, arm="r")
blue = butler.get("pfsArm", visit=68351, arm="b")
merged = butler.get("pfsMerged", visit=68351)
fiberIndex = 309  # fiberId = 339
fig, axes = plt.subplots()
axes.plot(red.wavelength[fiberIndex], red.flux[fiberIndex], "r-")
axes.plot(blue.wavelength[fiberIndex], blue.flux[fiberIndex], "b-")
plt.xlabel("Wavelength (nm)")
plt.ylabel("Arm flux (counts)")
plt.xlim((350, 1000))
plt.twinx()
plt.plot(merged.wavelength[fiberIndex], merged.flux[fiberIndex], "k-")
plt.ylabel("Merged flux (counts/nm)")
plt.show()
Comment by price [ 11/Jun/22 ]

Oh, the isr.doApplyGains config parameter should now be left alone. Attempts to switch it on (as we have been doing) will be met with an error.

Comment by price [ 11/Jun/22 ]

arnaud.lefur, RHL asked me to check with you that these changes don't invalidate your audit of the extractions. I think the important changes are:
1. The gains are now automatically applied; you should not (and cannot) set isr.doApplyGains=True.
2. The pfsMerged file now contains fluxes in units of electrons/nm. The normalisation has also changed to be in the same units, so that the normalised flux is consistent with what it was previously.

My opinion is that these changes should not change any of your conclusions, but please correct me if I am wrong.

Comment by hassan [ 11/Jun/22 ]

Code review complete. Only minor comment about units for pfsMerged files being in electron/nm rather than counts/nm.

Comment by hassan [ 11/Jun/22 ]

Given that arnaud.lefur will not have time to perform his audit for the next week or so with the June Engineering Run looming, and that this issue is blocking for Kiyoto Yabe, we will proceed to merge the proposed changes to master. Arnaud can perform his audit when he is next available, as part of a separate ticket. Thus any problems he may find will be addressed as part of that separate ticket.

Comment by price [ 11/Jun/22 ]

Merged.

Generated at Sat Feb 10 16:01:35 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.