[PIPE2D-655] Support all spectrograph detectors Created: 20/Nov/20  Updated: 21/Jan/21  Resolved: 25/Nov/20

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Blocker
Reporter: price Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to PIPE2D-702 Ensure full pipeline support for SM3 ... Done
Sprint: 2DDRP-2021 A
Reviewers: hassan

 Description   

fmadec wants to reduce b2 data, but the pipeline doesn't know about it. Add in all the detectors.



 Comments   
Comment by hassan [ 24/Nov/20 ]

Blocks SM2 activities at LAM.

Comment by price [ 24/Nov/20 ]

Need to:

  • Add detectors to the camera.yaml file.
  • Generate detectorMaps.
Comment by price [ 25/Nov/20 ]

hassan, I believe this is ready for review. I tested it out:

ingestPfsImages.py /projects/HSC/PFS/LAM --mode=link --config clobber=True register.ignore=True parse.pfsDesignId=0x0000010001001010 --pfsConfigDir=/projects/HSC/PFS/LAM/raw/pfsDesign -- "/projects/HSC/PFS/LAM/raw/*/PFLA*" "/projects/HSC/PFS/LAM/raw/*/sps/PFLA*"

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ constructPfsBias.py /projects/HSC/PFS/LAM --calib /projects/HSC/PFS/LAM/CALIB-price --rerun price/pipe2d-655 --id visit=23638..23652 --cores 10


RuntimeError: No defects available for dataId=DataId(initialdata={'visit': 23644, 'dateObs': '2020-11-20', 'site': 'L', 'category': 'A', 'arm': 'b', 'spectrograph': 2, 'field': 'NO_AVAILABLE_VALUE', 'ccd': 3, 'filter': 'b', 'expTime': 0.0, 'dataType': 'BIAS', 'taiObs': '2020-11-20T16:05:41.683', 'pfsDesignId': 1152921504606846976, 'slitOffset': 0.0}, tag=set())

--> Need to rebuild obs_pfs, to generate defects files.

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ rm -r /projects/HSC/PFS/LAM/CALIB-price/
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ mkdir /projects/HSC/PFS/LAM/CALIB-price

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ ingestPfsCalibs.py /projects/HSC/PFS/LAM/ --calib /projects/HSC/PFS/LAM/CALIB-price --validity=1800 --doraise --mode=copy /projects/HSC/PFS/LAM/rerun/price/pipe2d-655/BIAS/pfsBias-*.fits

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ constructPfsDark.py /projects/HSC/PFS/LAM --calib /projects/HSC/PFS/LAM/CALIB-price --rerun price/pipe2d-655 --id visit=23575 --cores 10
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ ingestPfsCalibs.py /projects/HSC/PFS/LAM/ --calib /projects/HSC/PFS/LAM/CALIB-price --validity=1800 --doraise --mode=copy /projects/HSC/PFS/LAM/rerun/price/pipe2d-655/DARK/pfsDark-2020-10-29-023575-b2.fits

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ constructFiberFlat.py /projects/HSC/PFS/LAM --calib /projects/HSC/PFS/LAM/CALIB-price --rerun price/pipe2d-655 --id visit=23592..23596 arm=b --cores 10

Breaks because it needs a detectorMap (though it shouldn't.... fixed).
But it looks like we don't really have any good dithered quartzes, so let's skip that.

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ constructFiberProfiles.py /projects/HSC/PFS/LAM --calib /projects/HSC/PFS/LAM/CALIB-price --rerun price/pipe2d-655 --id visit=23598 arm=b -c isr.doFlat=False --cores 10

Needs a detectorMap, for real this time (because we need to transfer the fiberId values from the detectorMap to the fiberProfiles).


import re
import astropy.io.fits
for arm in "brnm":
    with astropy.io.fits.open(f"detectorMap-sim-{arm}1.fits") as fits:
        ccd = dict(b=0, r=1, n=2, m=3)[arm]
        card = fits[0].header["CALIB_ID"]
        fits[0].header["CALIB_ID"] = re.sub("spectrograph=1 ccd=\d", f"spectrograph=2 ccd={ccd}", card)
        fits.writeto(f"detectorMap-sim-{arm}2.fits")

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-655 $ ingestPfsCalibs.py /projects/HSC/PFS/LAM --calib /projects/HSC/PFS/LAM/CALIB-price --validity=1800 --mode=copy drp_pfs_data/detectorMap/detectorMap-sim-*.fits

constructFiberProfiles works now.
Comment by price [ 25/Nov/20 ]

Merged to master.

Generated at Sat Feb 10 15:56:05 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.