[PIPE2D-1286] Please make it possible for reduceExposure to parallelise over visits and arms Created: 16/Aug/23 Updated: 24/Aug/23 Resolved: 24/Aug/23 |
|
| Status: | Done |
| Project: | DRP 2-D Pipeline |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Normal |
| Reporter: | rhl | Assignee: | price |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
reduceExposure.py doesn't parallelise over all the input dataIds, although the processing is initially independent. Please make it possible to run all visits/spectrographs/arms in parallel at least as far as a possible sequence point at full-focal-plane sky subtraction. As I believe that there is not yet a need for such sequence points, it would be reasonable to delay that scatter/gather work until it's needed (and then probably we'll do it in gen3 middleware anyway) |
| Comments |
| Comment by price [ 24/Aug/23 ] |
|
Merged to master. |