[DAMD-93] Clarify that hash is 63-bit Created: 07/Nov/20  Updated: 05/Jan/21  Resolved: 09/Nov/20

Status: Won't Fix
Project: Data Model
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: hassan
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to PIPE2D-652 createHash() should truncate to 63 bi... Won't Fix
Story Points: 1
Sprint: 2DDRP-2021 A

 Description   

Currently the datamodel.txt defines a hash as a 63-bit unsigned int, fitted into 64 bit signed integers:

https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L28

In various places I refer to a SHA-1, which is a strong 160-bit hash, as used by e.g. git
(https://en.wikipedia.org/wiki/SHA-1). We truncate these hashes to 63bits (so as to fit
in standard 64-bit signed integers). Sixty-three bits would produce up to 2^63 ~ 9e18 values.

This is fine and from recent discussions in the slack datamodel channel, in line with needs.

However, some later parts of the text in that file are inconsistent: https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L105

pfsVisitHash
An integer uniquely defining the set of visits contributing to a reduced spectrum;
this will be calculated as a SHA-1 truncated to 64 bits
See calculate_pfsVisitHash() in
python/pfs/datamodel/utils.py

(This should mention a truncation of the SHA-1 to 63 bits, and then fitted into a signed int).

Update the text accordingly.

There is a related problem raised in the discussion on slack, and that is the SHA generation function is not consistent with the 63-bit truncation:

https://github.com/Subaru-PFS/datamodel/blob/fa98c08c8ac839956f0d4e8489523e4898894a8b/python/pfs/datamodel/utils.py#L51

This will be fixed in a separate PIPE2D ticket.

 

 



 Comments   
Comment by hassan [ 09/Nov/20 ]

Withdrawn. Follow approach of using all 64 bits, and encoding into a signed 64-bit integer.

Generated at Sat Feb 10 15:34:10 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.