[DAMD-60] driver.py, makePfsObject.py and datamodel.ipynb need updating Created: 11/Jun/19  Updated: 11/Jul/19  Resolved: 11/Jul/19

Status: Done
Project: Data Model
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: hassan Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to DAMD-59 How can I make pfsArm object? Won't Fix
relates to PIPE2D-310 Modify pipeline flow to match design Done
Story Points: 2
Sprint: 2DDRP-2019 F
Reviewers: hassan

 Description   

Following the refactoring in Jan 2019 (PIPE2D-310) where a lot of the datamodel classes have been moved to drp.py, some utility code have not yet been updated accordingly so are currently not usable.

These include:

  • makePsfObject.py
  • driver.py
  • datamodel.ipynb

Please update them.



 Comments   
Comment by price [ 20/Jun/19 ]
  • makePfsObject.py is deliberately broken. I extracted it from another file when I built the end-to-end pipeline. I didn't delete it because one of its features (populating the fluxtbl) hasn't been incorporated into the pipeline. I think the process of resampling and coadding should belong in the algorithmic code of drp_stella rather than datamodel. I propose creating a ticket to implement the fluxtbl functionality in the pipeline.
  • bin/driver.py is awfully named. It appears to plot spectra from a PfsArm, then combine them into a PfsObject and plot them. I propose replacing this with a couple of bin scripts that will plot a PfsArm and PfsObject. As previously, I believe the algorithmic code doing the resampling and coadding should belong in drp_stella.
  • notebooks/datamodel.ipynb appears to have been a developmental aid. It duplicates much of the old code. I propose to delete it.
Comment by hassan [ 20/Jun/19 ]

Regarding makePfsObject: Yabe-san requires a tool to create a PfsObject from a PfsArm instance. See His comment in DAMD-59: https://pfspipe.ipmu.jp/jira/browse/DAMD-59?focusedCommentId=15646&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15646 . If your suggestion enables him to do that operation, that's fine (it's also fine to have such a tool in drp_stella).

Comment by price [ 20/Jun/19 ]

PIPE2D-438 is the ticket for implementing the fluxtbl. This will likely involve changes to the datamodel; I'll make a DAMD ticket once I know what those are.

Comment by price [ 21/Jun/19 ]

I tested out the plotting scripts, and they appear to work fine.

Comment by Kiyoto Yabe [ 25/Jun/19 ]

I'm a bit confused, but there appears to be "FLUXTBL" HDU in the current pfsObject, which contains the merged spectrum. So, are you going to rename FLUXTBL to FLUX and add FLUXTBL for the original spectra, right?

Related to the question above, I made pfsObject following codes in drp_stellar, but the HDU list of the generated fits file is :

HDU0 Primary

HDU1 FLUXTBL

HDU2 TARGET

HDU3 SKY

HDU4 OBSERVATIONS

HDU5 COVAR

HDU6 COVAR2

, which is different from the description in the datamodel.txt:

HDU #0 PDU

HDU #1 FLUX   

HDU #2 FLUXTBL

HDU #3 COVAR   

HDU #4 COVAR2   

HDU #5 MASK       

HDU #6 SKY         

HDU #7 CONFIG  

So, which one do we take?

 

 

 

Comment by price [ 26/Jun/19 ]

I screwed up the implementation of the pfsObject FITS HDUs when I implemented the end-to-end pipeline, since I didn't understand what FLUXTBL was meant for. So yes, I think I need to rename what is now FLUXTBL to FLUX, redesign FLUXTBL (the original design did not support multiple visits) and provide an implementation for populating it, and update datamodel.txt. Filed DAMD-63 for that.

Comment by hassan [ 11/Jul/19 ]

Changes acceptable.

Comment by price [ 11/Jul/19 ]

Merged to master.

Generated at Sat Feb 10 15:33:49 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.