https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-1405 Instrument control development <p>As discussed during the ICS/PFI telecon 2021-10-15: the second moment is dependent on the measurement algorithm used to measure AG spots. </p> <p>Please track this in the opDB, eg by an additional column in the <tt>agc_data</tt> table, and if necessary a separate table describing details of a given measurement algorithm. </p> INSTRM-1405

Provide the measurement algorithm used for AG spots

Task Normal Done Done Kiyoto Yabe hassan EngRun Fri, 15 Oct 2021 17:00:30 +0000 Wed, 17 Nov 2021 22:49:18 +0000 Wed, 17 Nov 2021 22:49:18 +0000 spt_operational_database 0 7 <p>The same as <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1406" title="Provide the measurement algorithm used for MCS spots in the opDB" class="issue-link" data-issue-key="INSTRM-1406"><del>INSTRM-1406</del></a>, I add a new column `measurement_algorithm` in `agc_exposure` table, assuming the algorithms are the same as those for MCS.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-python"> measurement_algorithm = Column(String, comment=<span class="code-quote">'Spot measurement algorithm (fast/slow)'</span>) </pre> </div></div> <p>Any comments are welcome. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=karr" class="user-hover" rel="karr">karr</a> <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hiro" class="user-hover" rel="hiro">Yoshida, Hiroshige</a></p> <p>We should probably do something more descriptive. Maybe "windowed" for my code and "sep" for the sextractor? I can change the keywords in the actor to match. </p> <p>In addition, I've uploaded a description of what methods are used to <a href="https://sumire.pbworks.com/w/browse/#:~:text=centroidMethod_karr.pdf" class="external-link" rel="nofollow">https://sumire.pbworks.com/w/browse/#:~:text=centroidMethod_karr.pdf</a></p> <p>Thank you for the document. I like descriptive keywords, but it's up to you. The column is `varchar`, so in terms of DB any keyword would be fine.</p> <p>We'll need to be able to use these keywords to get back to an algorithm in GitHub. The SHA1 needn't be provided, of course, as we'll be tracking the AG code version (which package is this code in?).</p> <p>We need to know about the flux measurement, the centroider, and the second moments. In general those are all different. If there are important configurations (e.g. the definitions of the windows used) how are those tracked? Using this mechanism or something else?</p> <p>Oh, and we also need the background levels in electrons over the bias. This is used for estimating the S/N (in conjunction with the second moments).</p> <p>Currently, versions of each actor used are not recorded in opDB. Is it a bad idea that we make `actor_version` (maybe GitHub tag can be used) and also `instdata_version` (again <em>pfs_instadata</em> tag can be used to track configuration yaml file) column in `agc_exposure` (and `mcs_exposure`) table? We may want to have the same mechanism in `sps_exposure`.</p> <p>Regarding the background level, we currently have a column `background` in `*<em>data` table, which is in _count_s in my first thought but can be in _electrons</em>. </p> <ol> <li>We should discuss how we manage versions on the summit; do you have a proposal (in SDSS we used eups)?  I think that we do probably need to import this information into opdb; <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> do you have opinions? The first thing that comes to mind is a table with versions and time ranges.</li> <li>I don't know if the AG works in electrons or DN, but I'd prefer the former.  We must know the gain at least approximately, so even if we're not flat fielding we could convert to electrons (it must be done internally to estimate the errors in the measurements – presumably we do get the errors along with the values).  Fluxes should probably be in electrons too, although if there are compelling reasons to use DN consistently that'd be OK providing the gain is available.  Does the AG code calculate the zero points?  If so, we need them in the opdb too.</li> </ol> <p>The ICS archiver and the FITS headers have per-actor versions, and actually a bit more detail than that: for any given actor we archive the version of the major ups packages (pfs_utils, tron_actorcore, etc), and for the FITS-generating actors we put the sub-package versions used by that actor into the header. W_VRxxxx cards are reserved for that purpose. Eups version unless the product directory is live, in which case git SHA-like.</p> <p>What to do for the opDb? Algorithm selection can be a runtime choice, so package versions might not be enough, but should be findable for any code we write. Not sure what to do: putting package versions in every exposure table feels crazy. Point to the headers?</p> <p>For now, I will add two string columns (`version_actor` and `version_instdata`) after discussion with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> and <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=rhl" class="user-hover" rel="rhl">rhl</a>.</p> <p>I think current changes are sufficient for the coming engineering run. Could you review that if possible?</p> <p>The column for centroid method looks good to me. </p> <p>Proposed changes look fine to me also.</p> <p>merged</p> Blocks INSTRM-1412 INSTRM-1429 Relates INSTRM-1406 Development Rank 0|zzs2x1: Reviewers hassan Sprint PreEngRun4 EngRun3Cleanup Story Points 2.0