https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-1258 Instrument control development <p>The raw data/fiberids/grandfibermap.xxx files are supposed to be wrapped by the code in fiberids.py. That still reads in the original JEG file, without all the changes in <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1195" title="Update grandfibermap following the recent discussions and requests" class="issue-link" data-issue-key="INSTRM-1195"><del>INSTRM-1195</del></a>, etc. Fix.</p> <p>A few methods might be used by ics_cobraCharmer but I suspect not many, since necessary columns were not present.</p> <p>The original short ("mf", etc) column names in the data file were replaced by more meaningful ones ("fiberInModuleId", etc). Should decide whether anyone cares; perhaps just drop them.</p> INSTRM-1258

Update the fiberids.py accessor for the grandfibermap file

Task Normal Done Done hassan cloomis SPS Tue, 27 Apr 2021 15:02:04 +0000 Thu, 3 Jun 2021 16:10:08 +0000 Thu, 6 May 2021 20:27:50 +0000 pfs_utils 0 3 <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Add proper module ID (no board) Add methods as needed cobraId vs. scienceFiberId? Why two, really? fiducials? Separate class, probably. Identify engineering/blank holes? </pre> </div></div> <p>While you are working on this, can you file a ticket for each of these and include their numbers in this list? We shouldn't have any TODOs that aren't associated with a jira ticket.</p> <p>[ That is from an old comment in fiberids.py, I believe. Looks familiar. Did generate requests on some if the recent GFM tickets. Comment should be removed/updated. ]</p> <p>From the <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1195" title="Update grandfibermap following the recent discussions and requests" class="issue-link" data-issue-key="INSTRM-1195"><del>INSTRM-1195</del></a>, etc. work:</p> <ul class="alternate" type="square"> <li>module ID added</li> <li>cobraId/fiberHoleId/scienceFiberId/fiberId better understood and made more sane.</li> <li>eng/blank rows added.</li> </ul> <p>That leaves the fiducial fibers and the "other methods". <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1259" title="Add better accessors for SuNSS fiber info" class="issue-link" data-issue-key="INSTRM-1259"><del>INSTRM-1259</del></a> addresses the "other methods".</p> <p>The fiducial fibers are described in another <tt>pfs_utils/data/fiberids/</tt> data file, which is read in and used directly by some part of the per-exposure/per-field fps/mcs calibration scheme. I do not know enough about that to say whether/how that API should be formalized.</p> <p>Main problem is that the reading in of the SuNSS fiberId was incorrect due to the wrong datatype for that column being specified. That has been fixed and a regression test added to trap situations in the future where changes are made to the data or corresponding fiber-related utilities.</p> <p>Changes are captured in <a href="https://github.com/Subaru-PFS/pfs_utils/pull/29" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/pfs_utils/pull/29</a></p> <p>See GH. I'm not sure that I like the masked array choice, and there's at least one typo.</p> <p>As of this morning the values of <tt>fiberIds.sunssFiberId</tt> do not correspond to the values in the file. Please change this back; I'd like to see <tt>"emp"</tt> or <tt>None</tt> not <tt>--</tt>. The same goes for other missing fields, please either return the value in the file or <tt>None</tt>/<tt>NaN</tt> and remove the use of masked arrays.</p> <p>In the latest push to the <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1258" title="Update the fiberids.py accessor for the grandfibermap file" class="issue-link" data-issue-key="INSTRM-1258"><del>INSTRM-1258</del></a> I've removed the use of <tt>MaskedArrays</tt>. But a problem remains in the handling of columns of integer type. As far as I can see, it is not possible to assign a value of <tt>None</tt> or <tt>np.nan</tt>. A legal integer value as the fill value is of course fine to do, and is currently 65535. If <tt>None</tt> or <tt>np.nan</tt> really required I need technical advice. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a>?</p> <p>Following slack discussion, a fill value of 65535 for unsigned integer fields is acceptable.</p> <p>If it's not too late, can we rename <tt>MISSING_VALUE_UINT16</tt> to e.g. <tt>MISSING_VALUE</tt>? The fact that that's a U16 number is irrelevant.</p> <p>Not too late, and one of Paul's comments. Ok for <tt>MISSING_VALUE_INTEGER</tt>? If so, already pushed.</p> <p>That's OK. The code will look like:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">if</span> fieldId == MISSING_VALUE_INTEGER: <span class="code-keyword">continue</span> </pre> </div></div> <p>and I personally think that the <tt>_INTEGER</tt> detracts from the readability as the type is irrelevant, but go with what you prefer.</p> <p>I would also prefer to remove <tt>_INTEGER</tt>, but problem is that this value is only used for the integer types. The float types use <tt>np.nan</tt>. The string types propagate <tt>-</tt> or <tt>emp</tt>.</p> <p>I've gone with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=rhl" class="user-hover" rel="rhl">rhl</a>'s suggestion and renamed to <tt>MISSING_VALUE</tt>. It's probably the most optimal name.</p> <p>Merged to master.</p> Blocks PIPE2D-833 Relates INSTRM-1249 INSTRM-1260 INSTRM-1259 INSTRM-1288 DAMD-110 Development Rank 0|02qpuc:i6001gr068fhw9qo4o0m8r Reviewers cloomis Sprint SM1PD-2021 A8 Story Points 3.0