[PIPE2D-897] pfsObject flux is not in nJy Created: 13/Sep/21  Updated: 24/Sep/21  Resolved: 24/Sep/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: Kiyoto Yabe Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 1
Sprint: 2DDRP-2021 A 8, 2DDRP-2021 A 9
Reviewers: hassan

 Description   

Looking at the weekly results at IPMU with `w.2021.37`, `pfsObject.flux` does not seem to be in nJy. Please confirm this and fix if this is true.



 Comments   
Comment by price [ 24/Sep/21 ]

Discovered a normalisation bug in flux calibration.

Comment by hassan [ 24/Sep/21 ]

Changes look OK to me. Did you look at the final pfsObject flux for an example file?

Comment by price [ 24/Sep/21 ]

Here's the log of my investigation:

>>> from lsst.daf.persistence import Butler
>>> butler = Butler("/scratch/pprice/jenkins/weekly/2021-09-19/process/rerun/weekly/pipeline/brn/pipeline/")

>>> pfsConfig = butler.get("pfsConfig", visit=47)
>>> from pfs.datamodel import TargetType
>>> pfsConfig.select(targetType=TargetType.FLUXSTD).objId
array([ 8, 13, 23, 53,  3, 26,  6, 52, 39, 54,  7, 33, 49, 41,  1, 37, 18,
       47, 40, 42, 56, 16, 17, 15, 46,  5, 25, 29, 57, 10,  2, 35, 31, 45,
       55,  4, 38, 58, 36, 44, 19, 11, 60, 34, 59, 30, 20,  9, 24, 51, 28,
       50, 32, 48, 22, 43, 21, 12, 14, 27])

Will work with objId=7

>>> pfsConfig.select(catId=0, objId=7).fiberFlux
[array([229086.765625])]
>>> pfsConfig.select(catId=0, objId=7).filterNames
[['i']]

So our object is i = 18 mag.

>>> pfsArm = butler.get("pfsArm", visit=47, arm="b").select(pfsConfig, objId=7, catId=0)
>>> select = (pfsArm.wavelength > 500) & (pfsArm.wavelength < 600)
>>> np.median(pfsArm.flux[select]/pfsArm.norm[select])
0.14605088737010558

>>> pfsMerged = butler.get("pfsMerged", visit=47).select(pfsConfig, objId=7, catId=0)
>>> select = (pfsMerged.wavelength > 500) & (pfsMerged.wavelength < 600)
>>> np.median(pfsMerged.flux[select])
5179.929950314969


>>> pfsSingle = butler.get("pfsSingle", catId=0, tract=0, patch="0,0", objId=7, visit=47)
>>> select = (pfsSingle.wavelength > 500) & (pfsSingle.wavelength < 600)
>>> np.median(pfsSingle.flux[select])
0.14152226134165563

>>> pfsObject = butler.get("pfsObject", catId=0, tract=0, patch="0,0", objId=7, nVisit=6, pfsVisitHash=0x7ace412dc05971ff)
>>> select = (pfsObject.wavelength > 500) & (pfsObject.wavelength < 600)
>>> np.median(pfsObject.flux[select])
0.0853505067399767

>>> pfsRef = butler.get("pfsReference", catId=0, tract=0, patch="0,0", objId=7)
>>> select = (pfsRef.wavelength > 500) & (pfsRef.wavelength < 600)
>>> np.median(pfsRef.flux[select])
213332.26323327963


The pfsSingle flux is the same as the pfsArm flux, so it must be terribly wrong since pfsArm is uncalibrated and pfsSingle should be in nJy.

Let's first look at the image to make sure there's a very bright trace where we expect.

>>> pfsConfig.select(catId=0, objId=7).fiberId
array([112], dtype=int32)
>>> detMap = butler.get("detectorMap", visit=47, arm="r")

There's definitely a brighter trace there. Not as bright as I thought it would be, but it's probably fine.

Found a normalisation bug in fluxCalibrate.

fluxCalibrate.py /scratch/pprice/jenkins/weekly/2021-09-19/process --calib=/scratch/pprice/jenkins/weekly/2021-09-19/process/CALIB --rerun=weekly/pipeline/brn/pipeline --doraise -j10 --id 'visit=47^48^49^53^54^57' --no-versions
coaddSpectra.py /scratch/pprice/jenkins/weekly/2021-09-19/process --calib=/scratch/pprice/jenkins/weekly/2021-09-19/process/CALIB --rerun=weekly/pipeline/brn/pipeline --doraise -j10 --id 'visit=47^48^49^53^54^57' --no-versions

Now:

>>> np.median(pfsSingle.flux[select])
213804.4453556766
>>> np.median(pfsObject.flux[select])
212780.95413807267
Comment by hassan [ 24/Sep/21 ]

Thanks Paul

Comment by price [ 24/Sep/21 ]

Merged.

Generated at Sat Feb 10 15:59:28 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.