[DAMD-58] Ensure that Object ID range covers the full 64 bits Created: 24/May/19  Updated: 26/Sep/19  Resolved: 26/Sep/19

Status: Done
Project: Data Model
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: hassan Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File PfsObjectId.py    
Issue Links:
Blocks
is blocked by PIPE2D-442 Upgrade to LSST stack v18.1.0 Done
Relates
relates to DAMD-8 Object IDs need to be 64bits, so 16 h... Done
relates to DAMD-56 Change pfsVisitHash formatting from %... Done
Story Points: 2
Sprint: 2DDRP-2019 F, 2DDRP-2019 G
Reviewers: hassan

 Description   

It was discovered while implementing DAMD-56 that 64 bit hash values could not be generated due to a limitation with PropertySet (see https://pfspipe.ipmu.jp/jira/browse/DAMD-56?focusedCommentId=15423&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15423). Object IDs, however, need to use the full 64-bit range.

It needs to be checked whether object IDs are prone to the same problem as hash values and if so, a correction needs to be made to the code.

 Comments   
Comment by hassan [ 30/May/19 ]

Attached python script demonstrates that when supplying an Object ID with the highest bit set, an error is raised with PropertySet.

Need to fix this in the LSST stack, making use of the <uint64_t> type. Quoting rhl in an email to hassan 2019-05-28 below:

We'll have to fix the LSST C++, but I don't think it's hard. Probably just a specialisation for <uint64_t>, but possibly a hook in the pybind11 layer too. Paul can do it.

Re-assigning ticket to price to fix.

Comment by price [ 11/Jul/19 ]

We need to fix this in LSST, and then we need to worry about how to get the changes into the stack we use.

Filed DM-20506: Allow PropertySet to handle 64-bit integers on the LSST side. I have a fix based on LSST 16.0 that I'll port.

Regarding how to get the changes into the stack we use, we essentially have two choices: upgrade our base stack (e.g., to hscPipe 7.x or LSST 18.x once it comes out with the fix we need), or find some way to inject the fixes into the stack we have now. I'm hoping to go with the latter, since this is a generic problem (wanting fixes in LSST that haven't been released yet).

Comment by price [ 11/Jul/19 ]

The LSST 18.0 release is imminent, but the LSST SQuARE team have agreed in principle to doing a fast turnaround on a patch release including the relevant work.

Comment by hassan [ 20/Jul/19 ]

See also https://jira.lsstcorp.org/browse/RFC-618 for updates as to the availability of LSST v18.1.0

Comment by price [ 22/Aug/19 ]

Only trivial changes are required once we upgrade to LSST 18.1.0.

Comment by price [ 22/Aug/19 ]

P.S. Deleted the tickets/DAMD-58 branch in drp_stella, which had my original attempts to fix this through monkey-patching. That didn't end up working, and LSST 18.1.0 solves the problem.

Comment by price [ 26/Sep/19 ]

Merged to master.

Generated at Sat Feb 10 15:33:48 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.