-
Type: Task
-
Status: Open (View Workflow)
-
Priority: Normal
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: None
-
Labels:
We have, I believe, agreed to use the opDb sps_annotation table to record bad PFS raw data files. Basically, if there is a row for the (visit, cam) and the data_flag column is not the "OK" value, the data should not be reduced.
Mineo-san's scripts already use the sps_annotation table to drive reductions. This change would apply to files which are not processed by that engine.
DRP uses a "headerFix" mechanism to track edits to header cards, which are applied when the file is read in: the raw data files are not themselves edited. This could be used to add a card indicating that the data are bad, in which case most of the machinery is already in place.
So:
- agree on how/when opDb data is propagated
- settle on a data quality card for the DRP.
- arrange for ingest or the first isr stage to skip bad files
- write opDb -> headerFix script
This is probably just an RFC ticket, until the right people check off on it...
- relates to
-
INSTRM-1202 Add command to create sps_annotations
- Done