[PIPE2D-845] Clarify the operations of reduceArc and reduceExposure Created: 24/May/21  Updated: 05/Aug/21  Resolved: 05/Aug/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PDF File PFS2DPipelineUserDocumentation.pdf    
Issue Links:
Relates
relates to PIPE2D-844 reduceExposure output detectorMap get... Done
Story Points: 2
Sprint: 2DDRP-2021 A5, 2DDRP-2021 A 6, 2DDRP-2021 A 7, 2DDRP-2021 A 8
Reviewers: hassan

 Description   

Following the discussion during the Princeton Group meeting 2021-05-17: please clarify the inputs, outputs and operations performed in the reduceExposure and reduceArc tasks in the context of arc line processing.

Specifically:

  • Specify where in the code the detectorMap is modified, and how (and is it overridden, stated in PIPE2D-844?)
  • Specify where the arcLines output is generated, and the inputs and algorithm used to generate this product. And again, if these are generated by reduceExposure, are they overridden in reduceArc??

This information can be provided as comments in this ticket, for the time being. If the approach is agreed within the Princeton Group, the design document should be updated.

 Comments   
Comment by price [ 15/Jun/21 ]

ReduceExposureTask is the general workhorse for reducing an exposure. For this reason, it is used as a sub-task by ReduceArcTask. ReduceExposureTask operates on a list of all arms of the same kind within a single visit (e.g., r1,r2,r3,r4). The following list of operations is correct following the changes made in PIPE2D-829:

  • Instrument signature removal and repair (CR masking, static bad pixels)
  • Wrangle spectral calibs:
    • fiberProfiles --> fiberTraces
    • Measure line centroids
    • detectorMap
      • Perform low-order adjustment
      • Write adjusted detectorMap as "detectorMap_used"
      • Regenerate fiberTraces
    • Measure line fluxes
  • Measure PSF and resulting LSF
  • 2D sky subtraction
  • Extract spectrum
    • Optional 2D continuum subtraction
  • Write outputs:
    • "calexp": exposure, possibly with 2D continuum subtracted
    • "pfsArmLsf": line-spread function
    • "sky2d": 2D sky subtraction model
    • "pfsArm": extracted spectra
    • "arcLines": line centroids+fluxes

ReduceArcTask runs ReduceExposureTask, collects the arcLines from different visits, fits a detectorMap and writes (as the "detectorMap" dataset). One wrinkle is that the file template for this is currently the same as for the "detectorMap_used" dataset. The filename does have to be the same (because of current limitations in the datamodel implementation), but we could write it to a different directory (but currently don't).

Comment by price [ 13/Jul/21 ]

This documentation needs to go somewhere "official", probably the "pipeline user" docs.

Also need to provide clear documentation on the steps needed to produce good calibs before running reduceExposure (bias, dark, flat, bootstrap, fiberProfile, reduceArc, fiberProfile, reduceArc).

Comment by price [ 13/Jul/21 ]

Describe the detectorMap model in some detail.

Comment by price [ 16/Jul/21 ]

I've updated the pipeline user doc: https://github.com/Subaru-PFS/drp_doc/pull/4

Comment by hassan [ 05/Aug/21 ]

Changes look good. Only minor comments added to pull request.

Comment by price [ 05/Aug/21 ]

Merged. hassan, could you please ensure the updated PDF is posted wherever the Right Place is?

Generated at Sat Feb 10 15:58:51 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.