[DAMD-93] Clarify that hash is 63-bit

Relates — Mon, 4 Jan 2021 20:23:06 +0000

Currently the datamodel.txt defines a hash as a 63-bit unsigned int, fitted into 64 bit signed integers:

https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L28

In various places I refer to a SHA-1, which is a strong 160-bit hash, as used by e.g. git
(https://en.wikipedia.org/wiki/SHA-1). We truncate these hashes to 63bits (so as to fit
in standard 64-bit signed integers). Sixty-three bits would produce up to 2^63 ~ 9e18 values.

This is fine and from recent discussions in the slack datamodel channel, in line with needs.

However, some later parts of the text in that file are inconsistent: https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L105

pfsVisitHash
An integer uniquely defining the set of visits contributing to a reduced spectrum;
this will be calculated as a SHA-1 truncated to 64 bits
See calculate_pfsVisitHash() in
python/pfs/datamodel/utils.py

(This should mention a truncation of the SHA-1 to 63 bits, and then fitted into a signed int).

Update the text accordingly.

There is a related problem raised in the discussion on slack, and that is the SHA generation function is not consistent with the 63-bit truncation:

https://github.com/Subaru-PFS/datamodel/blob/fa98c08c8ac839956f0d4e8489523e4898894a8b/python/pfs/datamodel/utils.py#L51

This will be fixed in a separate PIPE2D ticket.

PFS-JIRA

[DAMD-93] Clarify that hash is 63-bit