Uploaded image for project: 'DRP 2-D Pipeline'
  1. DRP 2-D Pipeline
  2. PIPE2D-423

Regular DRP processing of LAM and Subaru data

    XMLWordPrintable

    Details

    • Type: Story
    • Status: In Progress (View Workflow)
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      2DDRP-2019 E, 2DDRP-2021 A, 2DDRP-2021 A 2

      Description

      We need to support regular processing of LAM data, where the configuration is specified and persisted in order to track the inputs and reproduce those runs.

      A proposal from rhl in an email on 2019-04-03 to Naoki Yasuda is copied below:

      Craig and I have a proposal on how to do this in the short term, that can be generalised to work properly with an opDB later.
      
      I'll send around details when I have a chance, but basically we think it's quite easy to define a file (call it foo.yaml) that defines the inputs for all PFS calibrations.  Then a script that can be run in two modes:
      	1.  a. Write suitable entries in foo.yaml
      	    b. Run the script to generate the specified calibrations from foo.yaml
      	    c. Have a human check that the calibs are good (and/or run the quality assessment code)
      	    d. Run the script again (with different options) to ingest the new calibrations
      	    e. Push foo.yaml to git
      or
      	2.  a. Pull foo.yaml from git
      	    b. Run the script to generate and ingest the calibrations
      
      So we'd carry out the first procedure in one place (LAM?  Princeton?  IPMU?   NAOJ?), then carry out the second everywhere else.
      
      In operations, we'd generate the yaml file from the opDB.
      

      This should be implemented.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sogo.mineo sogo.mineo
                Reporter:
                hassan hassan
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: