[PIPE2D-359] Set up regular pipeline runs using real (LAM) data Created: 19/Feb/19  Updated: 21/May/19  Resolved: 01/Apr/19

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 4
Sprint: 2DDRP-2019 C
Reviewers: hassan

 Description   

Princeton are receiving LAM data regularly on the /tigress filesystem on their internal cluster. The 2DDRP pipeline should be run against those data frequently.

The raw LAM data consists of:

  1. Raw biases
  2. Raw darks
  3. Raw dithered flats
  4. Raw flats
  5. Raw arc images

The extent of the pipeline run should be such that all of these data types be reduced, with the resultant products being

  1. Biases
  2. Darks
  3. Flats
  4. FiberTraces
  5. Updated detectorMaps.

The pipeline can be ran manually at this stage, with the user providing the necessary visits. Automation will be the subject of a separate ticket when this is needed.

 Comments   
Comment by price [ 20/Mar/19 ]

Based on offline discussion with hassan, I understand that what is wanted is a script that will run the calib construction part of the pipeline, given a set of biases, darks, flats and arcs identified by the user. This can be adapted from the existing integration test script.

Comment by price [ 20/Mar/19 ]

hassan, could you please check that this is sufficient?

Comment by ncaplar [ 22/Mar/19 ]

Before I forget, numbers for the data that we currently have:

Neon data numbers:

bias: 12122..12136
dark: 12137..12166
flat: 12167..12171
ditheredFlats: 12172..12300

focused data:
12355..12360
12415...12477

defocused data:
12301..12354
12361..12414

Comment by ncaplar [ 22/Mar/19 ]

HgAr data numbers:
bias: 11485..11499
dark: 11500..11529
flat: 11530..11534
ditheredFlats: 11535..11663

focused data:
11664..11693
11748..11753
11808..11834

defocused data:
11694..11747
11754..11807

Comment by ncaplar [ 22/Mar/19 ]

Krypton data numbers

bias: 13128..13152 (but have no DATE-OBS) - DO NOT USE - see PIPE2D-424
dark: 13153..13182 (but have no DATE-OBS) - DO NOT USE - see PIPE2D-424
flat: 13183..13187 - DO NOT USE - see PIPE2D-424
ditheredFlats: 13188..13316 - DO NOT USE - see PIPE2D-424

focused data:
12922..12996
13051..13056
13111..113137

defocused data:
12997..13050
13057..13110

Comment by hassan [ 29/Mar/19 ]

Initial implementation viewed as done. Further work may be required as we more understanding of LAM needs etc.

Comment by price [ 01/Apr/19 ]

Merged to master.

Generated at Sat Feb 10 15:51:47 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.