Uploaded image for project: 'Data Model'
  1. Data Model
  2. DAMD-93

Clarify that hash is 63-bit

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Won't Fix (View Workflow)
    • Priority: Normal
    • Resolution: Won't Fix
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      2DDRP-2021 A

      Description

      Currently the datamodel.txt defines a hash as a 63-bit unsigned int, fitted into 64 bit signed integers:

      https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L28

      In various places I refer to a SHA-1, which is a strong 160-bit hash, as used by e.g. git
      (https://en.wikipedia.org/wiki/SHA-1). We truncate these hashes to 63bits (so as to fit
      in standard 64-bit signed integers). Sixty-three bits would produce up to 2^63 ~ 9e18 values.

      This is fine and from recent discussions in the slack datamodel channel, in line with needs.

      However, some later parts of the text in that file are inconsistent: https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L105

      pfsVisitHash
      An integer uniquely defining the set of visits contributing to a reduced spectrum;
      this will be calculated as a SHA-1 truncated to 64 bits
      See calculate_pfsVisitHash() in
      python/pfs/datamodel/utils.py

      (This should mention a truncation of the SHA-1 to 63 bits, and then fitted into a signed int).

      Update the text accordingly.

      There is a related problem raised in the discussion on slack, and that is the SHA generation function is not consistent with the 63-bit truncation:

      https://github.com/Subaru-PFS/datamodel/blob/fa98c08c8ac839956f0d4e8489523e4898894a8b/python/pfs/datamodel/utils.py#L51

      This will be fixed in a separate PIPE2D ticket.

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hassan hassan
                Reporter:
                hassan hassan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: