[INSTRM-1405] Provide the measurement algorithm used for AG spots Created: 16/Oct/21  Updated: 18/Nov/21  Resolved: 18/Nov/21

Status: Done
Project: Instrument control development
Component/s: spt_operational_database
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: Kiyoto Yabe
Resolution: Done Votes: 0
Labels: EngRun
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks INSTRM-1412 Integrate multiple centroiding method... Done
blocks INSTRM-1429 Update agcc database writes for new c... Done
Relates
relates to INSTRM-1406 Provide the measurement algorithm use... Done
Story Points: 2
Sprint: PreEngRun4, EngRun3Cleanup
Reviewers: hassan

 Description   

As discussed during the ICS/PFI telecon 2021-10-15: the second moment is dependent on the measurement algorithm used to measure AG spots.

Please track this in the opDB, eg by an additional column in the agc_data table, and if necessary a separate table describing details of a given measurement algorithm.



 Comments   
Comment by Kiyoto Yabe [ 28/Oct/21 ]

The same as INSTRM-1406, I add a new column `measurement_algorithm` in `agc_exposure` table, assuming the algorithms are the same as those for MCS.

    measurement_algorithm = Column(String, comment='Spot measurement algorithm (fast/slow)')

Any comments are welcome. karr Yoshida, Hiroshige

Comment by karr [ 28/Oct/21 ]

We should probably do something more descriptive. Maybe "windowed" for my code and "sep" for the sextractor? I can change the keywords in the actor to match. 

Comment by karr [ 28/Oct/21 ]

In addition, I've uploaded a description of what methods are used to https://sumire.pbworks.com/w/browse/#:~:text=centroidMethod_karr.pdf

Comment by Kiyoto Yabe [ 28/Oct/21 ]

Thank you for the document. I like descriptive keywords, but it's up to you. The column is `varchar`, so in terms of DB any keyword would be fine.

Comment by rhl [ 29/Oct/21 ]

We'll need to be able to use these keywords to get back to an algorithm in GitHub. The SHA1 needn't be provided, of course, as we'll be tracking the AG code version (which package is this code in?).

We need to know about the flux measurement, the centroider, and the second moments. In general those are all different. If there are important configurations (e.g. the definitions of the windows used) how are those tracked? Using this mechanism or something else?

Comment by rhl [ 29/Oct/21 ]

Oh, and we also need the background levels in electrons over the bias. This is used for estimating the S/N (in conjunction with the second moments).

Comment by Kiyoto Yabe [ 08/Nov/21 ]

Currently, versions of each actor used are not recorded in opDB. Is it a bad idea that we make `actor_version` (maybe GitHub tag can be used) and also `instdata_version` (again pfs_instadata tag can be used to track configuration yaml file) column in `agc_exposure` (and `mcs_exposure`) table? We may want to have the same mechanism in `sps_exposure`.

Regarding the background level, we currently have a column `background` in `*data` table, which is in _count_s in my first thought but can be in _electrons

Comment by rhl [ 11/Nov/21 ]
  1. We should discuss how we manage versions on the summit; do you have a proposal (in SDSS we used eups)?  I think that we do probably need to import this information into opdb; cloomis do you have opinions? The first thing that comes to mind is a table with versions and time ranges.
  2. I don't know if the AG works in electrons or DN, but I'd prefer the former.  We must know the gain at least approximately, so even if we're not flat fielding we could convert to electrons (it must be done internally to estimate the errors in the measurements – presumably we do get the errors along with the values).  Fluxes should probably be in electrons too, although if there are compelling reasons to use DN consistently that'd be OK providing the gain is available.  Does the AG code calculate the zero points?  If so, we need them in the opdb too.
Comment by cloomis [ 12/Nov/21 ]

The ICS archiver and the FITS headers have per-actor versions, and actually a bit more detail than that: for any given actor we archive the version of the major ups packages (pfs_utils, tron_actorcore, etc), and for the FITS-generating actors we put the sub-package versions used by that actor into the header. W_VRxxxx cards are reserved for that purpose. Eups version unless the product directory is live, in which case git SHA-like.

What to do for the opDb? Algorithm selection can be a runtime choice, so package versions might not be enough, but should be findable for any code we write. Not sure what to do: putting package versions in every exposure table feels crazy. Point to the headers?

Comment by Kiyoto Yabe [ 17/Nov/21 ]

For now, I will add two string columns (`version_actor` and `version_instdata`) after discussion with cloomis and rhl.

Comment by Kiyoto Yabe [ 17/Nov/21 ]

I think current changes are sufficient for the coming engineering run. Could you review that if possible?

Comment by karr [ 17/Nov/21 ]

The column for centroid method looks good to me. 

Comment by hassan [ 17/Nov/21 ]

Proposed changes look fine to me also.

Comment by Kiyoto Yabe [ 18/Nov/21 ]

merged

Generated at Sat Feb 10 16:35:07 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.