[PIPE2D-423] Regular DRP processing of LAM and Subaru data

Relates — Thu, 13 Jan 2022 14:39:30 +0000

We need to support regular processing of LAM data, where the configuration is specified and persisted in order to track the inputs and reproduce those runs.

A proposal from rhl in an email on 2019-04-03 to Naoki Yasuda is copied below:

Craig and I have a proposal on how to do this in the short term, that can be generalised to work properly with an opDB later.

I'll send around details when I have a chance, but basically we think it's quite easy to define a file (call it foo.yaml) that defines the inputs for all PFS calibrations.  Then a script that can be run in two modes:
	1.  a. Write suitable entries in foo.yaml
	    b. Run the script to generate the specified calibrations from foo.yaml
	    c. Have a human check that the calibs are good (and/or run the quality assessment code)
	    d. Run the script again (with different options) to ingest the new calibrations
	    e. Push foo.yaml to git
or
	2.  a. Pull foo.yaml from git
	    b. Run the script to generate and ingest the calibrations

So we'd carry out the first procedure in one place (LAM?  Princeton?  IPMU?   NAOJ?), then carry out the second everywhere else.

In operations, we'd generate the yaml file from the opDB.

This should be implemented.

PFS-JIRA

[PIPE2D-423] Regular DRP processing of LAM and Subaru data