[PIPE2D-578] Migrate the integration test to use YAML config files Created: 11/May/20  Updated: 05/Jan/21  Resolved: 10/Jun/20

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: sogo.mineo
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to PIPE2D-565 Write a script to generate a bash scr... Done
relates to PIPE2D-566 Write a YAML to do the same thing as ... Done
relates to PIPE2D-423 Regular DRP processing of LAM and Sub... In Progress
Story Points: 4
Sprint: 2DDRP-2021 A

 Description   

Following PIPE2D-565, bash processing scripts can be generated from YAML configuration files. Migrate the 2D DRP integration test to use this new mechanism.

 Comments   
Comment by sogo.mineo [ 08/Jun/20 ]

Is this a task to revise pfs_pipe2d/bin/pfs_integration_test.sh ?

Comment by price [ 08/Jun/20 ]

Yes.

Comment by sogo.mineo [ 08/Jun/20 ]

Is it essential that $DRP_STELLA_DATA_DIR falls back on "drp_stella_data" here?
https://github.com/Subaru-PFS/pfs_pipe2d/blob/master/bin/pfs_integration_test.sh#L111 

drp_stella_data=${DRP_STELLA_DATA_DIR:-drp_stella_data}

This line cannot be translated because the current generateCommands.py cannot interpret '${parameter:-word}'. If it is essential that an environment variable expansion can fall back on a constant, I have to update generateCommands.py.

Comment by price [ 10/Jun/20 ]

Wow, you flew through that!
The only comments I have are:

  • I find "integtest_*" a bit of an unwieldy name. I would prefer "integration_test_*" or just "test_*".
  • Please remove the commented-out lines in the YAML file.

I think we also need to do the same for the weekly (in pfs_pipe2d/weekly/process_weekly.sh), which may be a bit more challenging (there's more data, and it has m and n arms). Do you want to do that here, or shall we use a different ticket?

Comment by hassan [ 10/Jun/20 ]

Could we have a separate ticket for the weekly please sogo.mineo? It a big enough chunk of work and would be easier to track if it were a separate ticket.

Comment by sogo.mineo [ 10/Jun/20 ]

Thanks for review.
I chose "test_calib" and "test_science" for the block names
and merged this branch to master.

As for pfs_pipe2d/weekly/process_weekly.sh, I am creating another ticket.

Generated at Sat Feb 10 15:55:03 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.