[PIPE2D-380] Write fitted wavelength position and error to file Created: 04/Mar/19  Updated: 30/Aug/19  Resolved: 29/Aug/19

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0

Type: Task Priority: Normal
Reporter: hassan Assignee: Keigo Nakamura
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks PIPE2D-322 Write a function and task to determin... Won't Fix
Story Points: 3
Sprint: 2019 B, 2DDRP-2019 C, 2DDRP-2019 D, 2DDRP-2019 E, 2DDRP-2019 F, 2DDRP-2019 G
Reviewers: hassan

 Description   

The existing 2D DRP code (IdentifyLinesTask) already estimates positions and errors of spectral lines, based on a reference line list. We need to write this information in the case of arc data, to a separate file.

More specifically, the fiberId, wavelength, line positions and error (y and delta-y) are written to an ASCII or FITS file, along with additional information identifying the arc exposure (pfsDesignId and pfsConfigId).

In addition, the CalibrateWavelengthsTask uses the information from IdentifyLinesTask to determine a wavelength solution for the detectorMap, flagging lines that are used as part of the fitting, or those that are reserved. That task also provides a revised estimate and error of the wavelength solution.

This additional information should also be written to the same file (there is some denormalization here but it makes more practical sense to carry both sets of information in the same file).

I suggest updating the method reduceArcTask.write() to do this, as it appears that most of the information is available there.



 Comments   
Comment by price [ 04/Mar/19 ]

This should be done through the data butler.

Comment by price [ 05/Mar/19 ]

Here's what's needed to implement this with the data butler:
1. Devise a class that will hold the data. The class should implement the following interfaces:

class WavelengthFitStatistics:
    @classmethod
    def readFits(cls, filename):
        """Read from file

        The method name says "readFits", but this doesn't actually need to be FITS.
        If the name bothers you, this can forward to a method with a better name.
        """
        # Read data from file, then:
        return cls(various, arguments)

    def writeFits(self, filename):
        """Write to file

        The method name says "writeFits", but this doesn't actually need to be FITS.
        If the name bothers you, this can forward to a method with a better name.
        """
        # Write data to file

2. Add an entry to the datasets section of obs_pfs/policy/PfsMapper.yaml. Pattern it on an existing entry, like the one for fluxCal.

3. To write the data, construct the object and then pass the object to the data butler (butler.put(WavelengthFitStatistics(data), "wavelengthStats", dataId)) or data butler reference (dataRef.put(WavelengthFitStatistics(data), "wavelengthStats")). The name (e.g, "wavelengthStats") should match the entry made in step 2.

Comment by hassan [ 20/May/19 ]

Re-assigned temporarily to price. He would provide initial implementation, that would be updated by Keigo Nakamura if required.

Comment by price [ 23/May/19 ]

Handing this back to Keigo Nakamura after putting together an initial implementation on tickets/PIPE2D-380. I moved the former implementation to u/keigo/PIPE2D-380.

Questions are welcome, but I'm going to be in the back woods for the next week, so my internet access will be spotty.

price@MacBook:~/pfs/drp_stella (tickets/PIPE2D-380=) $ git sub
commit 7d064a37b3c867066c22cf476177f7a63ac2a5c1 (HEAD -> tickets/PIPE2D-380, origin/tickets/PIPE2D-380)
Author: Paul Price <price@astro.princeton.edu>
Date:   Wed May 22 20:26:45 2019 -0400

    save wavelength fit data
    
    A new class, WavelengthFitData, saves the wavelength fit data
    (centroid in pixels, fit wavelength, actual wavelength) for
    calculating statistics and for later QA.

 python/pfs/drp/stella/calibrateWavelengthsTask.py | 203 +++++++++++++++++++++-
 python/pfs/drp/stella/reduceArcTask.py            |   3 +-
 tests/test_WavelengthFitData.py                   | 117 +++++++++++++
 3 files changed, 321 insertions(+), 2 deletions(-)


price@MacBook:~/pfs/obs_pfs (tickets/PIPE2D-380=) $ git sub
commit 0a7c056a37ae247950063af9b45f78c5ac3b7edd (HEAD -> tickets/PIPE2D-380, origin/tickets/PIPE2D-380)
Author: Paul Price <price@astro.princeton.edu>
Date:   Wed May 22 15:38:28 2019 -0400

    PfsMapper: add wlFitData dataset
    
    For storing wavelength fit data.

 policy/PfsMapper.yaml | 10 ++++++++++
 1 file changed, 10 insertions(+)
Comment by price [ 23/Aug/19 ]

I believe this is ready to merge.

Comment by price [ 29/Aug/19 ]

Merged to master.

Generated at Sat Feb 10 15:52:30 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.