[DAMD-63] Redesign and implement pfsObject FluxTbl Created: 26/Jun/19 Updated: 25/Oct/19 Resolved: 25/Oct/19 |
|
| Status: | Done |
| Project: | Data Model |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Normal |
| Reporter: | price | Assignee: | price |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Story Points: | 3 | ||||||||||||||||
| Sprint: | 2DDRP-2019 F, 2DDRP-2019 G, 2DDRP-2019 H | ||||||||||||||||
| Reviewers: | hassan | ||||||||||||||||
| Description |
|
I screwed up the implementation of the pfsObject FITS HDUs when I implemented the end-to-end pipeline, since I didn't understand what FLUXTBL was meant for. We need to rename what is now FLUXTBL to FLUX, redesign FLUXTBL (the original design did not support multiple visits) and provide an implementation for populating it, and update datamodel.txt. |
| Comments |
| Comment by price [ 26/Jun/19 ] |
|
hassan points out that the CONFIG and MASK extensions have been removed to. CONFIG has been removed in favour of pfsConfig, and I believe MASK has been subsumed into what will be FLUX. Let's fix these up in datamodel.txt at the same time. |
| Comment by price [ 30/Jul/19 ] |
|
Here's a plot of a coadd (black) and the fluxTable contents (purple, yellow, red, cyan).
>>> from lsst.daf.persistence import Butler >>> butler = Butler("INTEGRATION/rerun/integration/pipeline") >>> obj = butler.get("pfsObject", catId=1, tract=0, patch="0,0", objId=0x5d, nVisit=4, pfsVisitHash=0x42cfc4af8cc94582) >>> fig, axes = obj.fluxTable.plot(show=False) >>> axes.plot(obj.wavelength, obj.flux, 'k-') >>> fig.show() |
| Comment by rhl [ 30/Jul/19 ] |
|
I am confused by the comment about CONFIG. It is not replaced by the pfsConfig file, it was explicitly denormalised for the benefit of end users. I also don't understand the remark about multiple epochs. The pfsObject file is the combined spectrum; if you want the input visits you need to go back to the pfsArm files. |
| Comment by price [ 30/Jul/19 ] |
|
Now I'm confused too. You say that pfsObject is a combined spectrum, but a spectrum that has been coadded from multiple observations doesn't (necessarily) have a single top-end configuration. So what are you expecting to find in the CONFIG extension? What is the point of having the FLUXTBL as opposed to just the flux vector of the coadd? My understanding is:
A coadded spectrum is a convenient representation of the spectrum of an
object with multiple observations, but the act of coaddition necessarily
involves a reduction in the amount of data through resampling in wavelength
and replacing a distribution of fluxes with a statistical representation.
The intention of a ``FluxTable`` is to provide the full set of original data
so that the user can have the option to inspect and fit the observations
themselves rather than a reduced, resampled and censored version of the
observations. This may be useful for identifying artifacts that have leaked
into the coadds (e.g., cosmic rays) or transient spectral features, or for
doing Hogg-ish Bayesian fitting.
|
| Comment by price [ 06/Aug/19 ] |
|
I believe this is now ready for people to look at. rhl and Kiyoto Yabe have expressed an interest, but I welcome comments from anyone. |
| Comment by price [ 02/Oct/19 ] |
|
I believe I have addressed concerns about changes to the datamodel:
Please have another look. |
| Comment by price [ 25/Oct/19 ] |
|
Merged to master. |