[PIPE2D-828] Port LSST DM-29117 ticket for handling 32/64 bit keywords Created: 01/May/21 Updated: 08/Jun/21 Resolved: 08/Jun/21 |
|
| Status: | Done |
| Project: | DRP 2-D Pipeline |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Normal |
| Reporter: | rhl | Assignee: | price |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Story Points: | 1 | ||||||||
| Sprint: | 2DDRP-2021 A 4, 2DDRP-2021 A5 | ||||||||
| Reviewers: | hassan | ||||||||
| Description |
|
A command like: butler.get('raw', visit=45777, arm='r', filter='r') fails as the type if the W_PFDSGN keyword changes when we patch the header to insert the correct values. Please pull the fix from the LSST side of things. If this is not practical, I have a horrible hack in pfsMapper.py which we could commit. |
| Comments |
| Comment by rhl [ 01/May/21 ] |
|
I told you it was horrible. I don't see a better way than save/maybe restore. diff --git a/python/lsst/obs/pfs/pfsMapper.py b/python/lsst/obs/pfs/pfsMapper.py index dd33d4d..05995e7 100644 --- a/python/lsst/obs/pfs/pfsMapper.py +++ b/python/lsst/obs/pfs/pfsMapper.py @@ -58,6 +58,22 @@ class PfsRawVisitInfo(MakeRawVisitInfo): return self.offsetDate(dateObs, 0.5*exposureTime) +def fix_header_safe(md, translator_class=PfsTranslator): + """A version of fix_header that works around DM-29117""" + dangerousKeys = ["W_PFDSGN"] # keys that can trigger DM-29117 + saved = {} + for k in dangerousKeys: + if k in md: + saved[k] = md.get(k) + md.remove(k) + + fix_header(md, translator_class=PfsTranslator) + + for k in dangerousKeys: + if k not in md and k in saved: + md[k] = saved[k] + + class PfsMapper(CameraMapper): """Provides abstract-physical mapping for PFS data""" packageName = "obs_pfs" @@ -219,7 +235,9 @@ class PfsMapper(CameraMapper): exp = super(PfsMapper, self).std_raw(item, dataId) md = exp.getMetadata() - fix_header(md, translator_class=PfsTranslator) + + fix_header_safe(md, translator_class=PfsTranslator) + try: dataVersion = int(md.get('W_VERSIONS_FPGA'), 16) except Exception: @@ -248,7 +266,7 @@ class PfsMapper(CameraMapper): item : `lsst.daf.base.PropertyList` The modified raw metadata. """ - fix_header(item, translator_class=PfsTranslator) + fix_header_safe(item, translator_class=PfsTranslator) return item def std_fiberProfiles(self, item, dataId): |
| Comment by hassan [ 07/Jun/21 ] |
|
Link to LSST DM-29117 ticket: https://jira.lsstcorp.org/browse/DM-29117 |
| Comment by price [ 08/Jun/21 ] |
|
Brought in code from LSST. |
| Comment by price [ 08/Jun/21 ] |
|
And it works: >>> exp = butler.get("raw", visit=45777, arm="r", filter="r") >>> exp.getMetadata().get("W_PFDSGN") 3735928559 |
| Comment by hassan [ 08/Jun/21 ] |
|
Changes look fine to me. Only minor comments. |
| Comment by hassan [ 08/Jun/21 ] |
|
Also checked with
python -c "import lsst.obs.pfs.dm29117; import lsst.daf.base as dafBase; md = dafBase.PropertyList(); md['aa'] = 1; md['aa'] = 3735928559"
which passed successfully. |
| Comment by price [ 08/Jun/21 ] |
|
Merged. I also added a commit that allows you to leave off the filter="r" in the buter.get call. >>> from lsst.daf.persistence import Butler >>> butler = Butler("/projects/HSC/PFS/Subaru") CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3 >>> butler.get("raw", visit=45777, arm="r") <lsst.afw.image.exposure.exposure.ExposureU object at 0x2b4a13f32618> |