https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/DAMD-146 Data Model <p>In the current openuse policy, the PFS fibers will be shared among more then one programs. It is efficient to add a column for the proposal id(s) in pfsDesign and hence pfsConfig in order to  know which target is for which program (and manage access during their priority period).</p> <p>Details (column name, data format) will be added.</p> DAMD-146

Add a column for proposal ID in DESIGN/CONFIG table in pfsDesign/pfsConfig files

Task Normal Done Done monodera yuki.moritani EngRun Wed, 5 Apr 2023 05:37:52 +0000 Thu, 14 Sep 2023 22:12:41 +0000 Thu, 14 Sep 2023 22:12:41 +0000 0 5 <p>I think that the necessary information to identify proposals and objects for tracking progress is <tt>ob&#95;code</tt> (string) and <tt>proposal&#95;id</tt> (string) (or <tt>group&#95;id</tt> (string)) from <tt>targetdb</tt>. Which is easier, to add columns to the existing extension (perhaps 1st extension) or to add a new extension including such operation-related columns in pfsDesign and pfsConfig?</p> <p>I suggest we put it in the existing main extension with everything else, and give it a default value if it's not present (old files).</p> <p>So, you'd add a proposal_id attribute with a value per fiber, so 2458 values, is that right ?</p> <p>I believe we need to associate a proposal with each fiber. I think the options are to have a <tt>proposalId</tt> for each fiber (in the same way each fiber has an ra, dec, etc), or we could write a new extension with <tt>proposalId</tt> associated with a variable-length array of <tt>fiberId</tt>. I like the former, because it naturally forces a proposal to be associated with each fiber, whereas the latter requires validation to check that each fiber has a proposal.</p> <p>A <tt>(proposalID, obCode)</tt> pair defines a unique target (ra/dec, requested total exposure time, instrument configuration, observing condition, etc.) and I think that adding these two keys would simplify tracking the observation progress for each target. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=eric" class="user-hover" rel="eric">eric</a>, do you have any comments?</p> <p>I'm also fine to add columns to the existing extension rather than adding a new one. </p> <p>Could you please explain what <tt>obCode</tt> is?</p> <p>Right, the first proposal makes sense to me too, it's consistent with what we did so far.</p> <p>&gt; <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=eric" class="user-hover" rel="eric">eric</a>, do you have any comments?</p> <p>I think that would simplify things a lot for tracking queue progress if we can see the proposal and ob code directly from the op-db.</p> <p>&gt; Could you please explain what <tt>obCode</tt> is?</p> <p>The ob_code is a string chosen by the PI for a particular OB (Observation Block), which is the "basic unit of observation" in the Subaru style of queue observing.  OB codes identify the OB being observed and must be unique within each proposal, thus the combination of proposal and ob_code is a unique OB (another observer might choose the same code).  The codes are chosen by the PI so that they may have meaningful names for planning their observations.</p> <p> </p> <p> </p> <p>Here is a pull request for this ticket.</p> <p><a href="https://github.com/Subaru-PFS/datamodel/pull/118" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/datamodel/pull/118</a></p> <p>As a combination of (proposalId, obCode) must be unique, some validation code must be present, but not yet implemented in the PR. In principle, this validation must be done when we ingest targets to the target database and the unique constraint has already been implemented there.</p> <p>I'm adding a backward compatibility when <tt>proposalId</tt> and <tt>obCode</tt> do not exist in the case of previous pfsDesign and pfsConfig files. I'm adding "dummy" for these cases, but do you have any preferences for it? For example, <tt>proposalId = None</tt> and <tt>obCode = fiberId</tt> may also make sense. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=eric" class="user-hover" rel="eric">eric</a></p> <p>&gt; do you have any preferences for it? </p> <p>It may be nice to preserve the proposalId, IMO.  In such case we could then infer that an obCode == None is a classical observation, and the proposalId is valid. Just my .02c</p> <p>We can't know which proposal ID is assigned for each fiber from a design/config file made, e.g., a year ago as there are no columns for these keys. That's why I thought we need to provide a dummy value. </p> <p>I think in that case that None would be appropriate for proposalId.</p> <p>Is there no other field for fiberId?  I think overloading a field with a value that is in a completely different domain will be potentially confusing.  These things then become more complicated to handle in software.</p> <p>The current master branch of datamodel lists the following items in the DESIGN table (1st extension of pfsDesign file).</p> <p> <cite>fiberId 32-bit int</cite><br/> ?? catId 32-bit int??<br/> ?? tract 32-bit int??<br/> ?? patch string??<br/> ?? objId 64-bit int??<br/> ?? ra 64-bit float (degrees)??<br/> ?? dec 64-bit float (degrees)??<br/> ?? targetType 32-bit int (enumerated type: e.g. SCIENCE,SKY,FLUXSTD)??<br/> ?? fiberStatus 32-bit int (enumerated type: e.g. GOOD,BROKENFIBER,BLOCKED,BLACKSPOT)??<br/> ?? pfiNominal pair of 32-bit floats (millimeters on the PFI)??</p> <p>The fiberId is already present, and I'm adding proposalId and obCode in the table.</p> <p>After having a chat with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=eric" class="user-hover" rel="eric">eric</a> , we agreed to put a dummy string ("N/A") rather than None to indicate the datatype explicitly. I'll make a change and make a commit. I guess I'll also need to be consistent in dummyCableB pfsDesign (and it should be straightforward).</p> <p>Commits have been merged into master branches of datamodel, drp_stella, and pfs_utils.</p> Development Rank 0|zzszf9: Sprint Eng13Oct