[PIPE2D-906] Fix weekly breakage in reduceExposure Created: 27/Sep/21  Updated: 30/Sep/21  Resolved: 30/Sep/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: price Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 2
Sprint: 2DDRP-2021 A 9
Reviewers: hassan

 Description   

In the past weekly, reduceExposure died:

Traceback (most recent call last):
 File "/scratch/pprice/jenkins/weekly/2021-09-26/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.39/bin/reduceExposure.py", line 25, in <module>
   ReduceExposureTask.parseAndRun()
 File "/scratch/pprice/jenkins/weekly/2021-09-26/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 603, in parseAndRun
   resultList = taskRunner.run(parsedCmd)
 File "/scratch/pprice/jenkins/weekly/2021-09-26/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
   resultList = list(mapFunc(self, targetList))
 File "/scratch/pprice/jenkins/weekly/2021-09-26/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 45, in _runPool
   return pool.map_async(function, iterable).get(timeout)
 File "/scratch/pprice/jenkins/weekly/2021-09-26/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 657, in get
   raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x2b929f66dd30>'. Reason: 'TypeError("can't pickle lsst.pex.exceptions.exceptions.LengthError objects")'


 Comments   
Comment by price [ 29/Sep/21 ]

The immediate problem was that the header fix database didn't know whether it was fixing Subaru, LAM or simulator data, so fixes intended for Subaru were being applied to the weekly.

Once that was fixed, the QA thresholds needed a bit of tweaking. My guess is that we were previously identifying a Ne arc as Kr, so there was a lot of rejection, leading to abnormally low RMS in the wavelength solution.

Comment by hassan [ 29/Sep/21 ]

Changes overall look fine. One minor comment related to docStrings for drp_stella changes (https://github.com/Subaru-PFS/drp_stella/pull/222).

Comment by price [ 30/Sep/21 ]

Merged.

Generated at Sat Feb 10 15:59:33 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.