<!-- 
RSS generated by JIRA (8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b) at Sat Feb 10 15:34:12 JST 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>PFS-JIRA</title>
    <link>https://pfspipe.ipmu.jp/jira</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>8.3.4</version>
        <build-number>803005</build-number>
        <build-date>13-09-2019</build-date>
    </build-info>


<item>
            <title>[DAMD-94] Fix discrepancy between createHash and datamodel.txt</title>
                <link>https://pfspipe.ipmu.jp/jira/browse/DAMD-94</link>
                <project id="10400" key="DAMD">Data Model</project>
                    <description>&lt;p&gt;Currently the &lt;tt&gt;datamodel.txt&lt;/tt&gt; defines a hash as a 63-bit unsigned int, fitted into 64 bit &lt;em&gt;signed&lt;/em&gt; integers:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L28&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://github.com/Subaru-PFS/datamodel/blob/7af03e7b2adba3b5e190995b54296d70683e6d17/datamodel.txt#L28&lt;/a&gt;&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;In various places I refer to a SHA-1, which is a strong 160-bit hash, as used by e.g. git&lt;br/&gt;
 (&lt;a href=&quot;https://en.wikipedia.org/wiki/SHA-1&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/SHA-1&lt;/a&gt;). We truncate these hashes to 63bits (so as to fit&lt;br/&gt;
 in standard 64-bit signed integers). Sixty-three bits would produce up to 2^63 ~ 9e18 values.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;This is currently inconsistent with the &lt;tt&gt;datamodel.utils.createHash()&lt;/tt&gt; function, where a 64-bit hash is generated:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/Subaru-PFS/datamodel/blob/fa98c08c8ac839956f0d4e8489523e4898894a8b/python/pfs/datamodel/utils.py#L51&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://github.com/Subaru-PFS/datamodel/blob/fa98c08c8ac839956f0d4e8489523e4898894a8b/python/pfs/datamodel/utils.py#L51&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Fix this discrepancy following the proposal by Sogo Mineo and Craig Loomis in the datamodel channel 2020-11-06, by updating the &lt;tt&gt;datamodel.txt&lt;/tt&gt; text mentioned above, such that a &lt;b&gt;64-bit&lt;/b&gt; hash is generated, in line with &lt;tt&gt;createHash&lt;/tt&gt;, and that this hash can be fitted into a standard 64-bit &lt;b&gt;signed&lt;/b&gt; integer. &lt;/p&gt;

&lt;p&gt;This will allow identifiers such as the &lt;tt&gt;pfsDesignId&lt;/tt&gt;, which use that hash, to be stored in the opDB Postgres database, using a standard &lt;tt&gt;bigint&lt;/tt&gt; or &lt;tt&gt;int8&lt;/tt&gt; data type, without need for additional conversion routines, as discussed in &lt;a href=&quot;https://pfspipe.ipmu.jp/jira/browse/INSTRM-1096&quot; title=&quot;Add {{pfs_visit.pfs_design_id}} column&quot; class=&quot;issue-link&quot; data-issue-key=&quot;INSTRM-1096&quot;&gt;&lt;del&gt;INSTRM-1096&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;


&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="15018">DAMD-94</key>
            <summary>Fix discrepancy between createHash and datamodel.txt</summary>
                <type id="3" iconUrl="https://pfspipe.ipmu.jp/jira/secure/viewavatar?size=xsmall&amp;avatarId=10518&amp;avatarType=issuetype">Task</type>
                                            <priority id="10000" iconUrl="https://pfspipe.ipmu.jp/jira/images/icons/priorities/medium.svg">Normal</priority>
                        <status id="10002" iconUrl="https://pfspipe.ipmu.jp/jira/images/icons/statuses/generic.png" description="The issue is resolved, reviewed, and merged">Done</status>
                    <statusCategory id="3" key="done" colorName="green"/>
                                    <resolution id="10000">Done</resolution>
                                        <assignee username="hassan">hassan</assignee>
                                    <reporter username="hassan">hassan</reporter>
                        <labels>
                    </labels>
                <created>Mon, 9 Nov 2020 13:54:21 +0000</created>
                <updated>Mon, 4 Jan 2021 20:23:06 +0000</updated>
                            <resolved>Fri, 13 Nov 2020 20:04:46 +0000</resolved>
                                                                        <due></due>
                            <votes>0</votes>
                                    <watches>2</watches>
                                                                <comments>
                            <comment id="18032" author="hassan" created="Mon, 9 Nov 2020 13:57:32 +0000"  >&lt;p&gt;Other identifiers such as the &lt;tt&gt;objId&lt;/tt&gt; are also affected. Hassan is investigating whether the Gaia DR2 object identifier, the &lt;em&gt;sourceId&lt;/em&gt;, can be stored in the positive range of a 64-bit signed int to avoid possible confusion. Yabe-san is checking the HSC ID.&lt;/p&gt;</comment>
                            <comment id="18037" author="sogo.mineo" created="Tue, 10 Nov 2020 05:22:14 +0000"  >&lt;p&gt;I think the hexadecimal notation of the 64bit hash will be kept unsigned as it is now, e.g. &lt;tt&gt;0xfedcba9876543210&lt;/tt&gt;.&lt;br/&gt;
What will the signed value for &lt;tt&gt;x = 0xfedcba9876543210&lt;/tt&gt;?&lt;br/&gt;
I think &lt;tt&gt;fits_cast_unsigned_to_signed&amp;#40;x&amp;#41;&lt;/tt&gt; is currently used.&lt;br/&gt;
What I meant in slack was &lt;tt&gt;cplusplus_cast_unsigned_to_signed&amp;#40;x&amp;#41;&lt;/tt&gt;.&lt;br/&gt;
I don&apos;t have preference but it should be determined which to use.&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
def fits_cast_unsigned_to_signed(x):
    &lt;span class=&quot;code-keyword&quot;&gt;return&lt;/span&gt; x - 0x8000_0000_0000_0000

def cplusplus_cast_unsigned_to_signed(x):
    &lt;span class=&quot;code-keyword&quot;&gt;return&lt;/span&gt; x - ((x &amp;amp; 0x8000_0000_0000_0000) &amp;lt;&amp;lt; 1)
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;</comment>
                            <comment id="18047" author="hassan" created="Thu, 12 Nov 2020 16:39:53 +0000"  >&lt;p&gt;Following subsequent discussions with &lt;a href=&quot;https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis&quot; class=&quot;user-hover&quot; rel=&quot;cloomis&quot;&gt;cloomis&lt;/a&gt; and &lt;a href=&quot;https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=sogo.mineo&quot; class=&quot;user-hover&quot; rel=&quot;sogo.mineo&quot;&gt;sogo.mineo&lt;/a&gt;: generating a hash that is truncated to from the original 160 bit SHA-1 to 64-bits, compared with one truncated to 63 bits, would cause more confusion and problems than it helps. &lt;/p&gt;

&lt;p&gt;For example, if the resultant 64-bit hash is carried internally as a 64-bit signed integer (for example when read from the opDB, where the column type is 64-bit signed), then approx 50% of all generated hashes would result in a negative signed integer value being read. Care then needs to be taken when using these, for example when writing out such has values as hex representations in file names, as performed by the LSST Gen2 Butler (tests show in fact that the Butler would raise an error in such situations).&lt;/p&gt;

&lt;p&gt;It appears to be much simpler and safer to truncate the SHA to 63 bits. This way, all hash values will be positive. &lt;/p&gt;</comment>
                            <comment id="18051" author="hassan" created="Fri, 13 Nov 2020 20:04:46 +0000"  >&lt;p&gt;merged to master (d472332)&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10000">
                    <name>Blocks</name>
                                            <outwardlinks description="blocks">
                                        <issuelink>
            <issuekey id="15004">INSTRM-1096</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10500" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10010" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0|02qpro:2</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10005" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="93">2DDRP-2021 A</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10002" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                        </customfields>
    </item>
</channel>
</rss>