[PIPE2D-892] Fix misshapen array in fluxCalibrate Created: 07/Sep/21  Updated: 08/Sep/21  Resolved: 08/Sep/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: price Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: 2DDRP-2021 A 8
Reviewers: hassan

 Description   

The weekly failed:

+ fluxCalibrate.py /scratch/pprice/jenkins/weekly/2021-09-05/process --calib=/scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB --rerun=weekly/pipeline/brn/pipeline --doraise -j10 --id 'visit=47^48^49^53^54^57'
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
root INFO: Running: /scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/bin/fluxCalibrate.py /scratch/pprice/jenkins/weekly/2021-09-05/process --calib=/scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB --rerun=weekly/pipeline/brn/pipeline --doraise -j10 --id visit=47^48^49^53^54^57
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 121, in worker
   result = (True, func(*args, **kwds))
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
   return list(map(*args))
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 385, in __call__
   result = self.runTask(task, dataRef, kwargs)
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
   return task.runDataRef(dataRef, **kwargs)
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/python/pfs/drp/stella/fluxCalibrate.py", line 127, in runDataRef
   self.calculateCalibrations(mergedFluxCal, references)
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/python/pfs/drp/stella/fluxCalibrate.py", line 200, in calculateCalibrations
   merged.covar[0] += self.config.sysErr*merged.flux  # add systematic error
ValueError: operands could not be broadcast together with shapes (3,11376) (60,11376) (3,11376) 
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/bin/fluxCalibrate.py", line 3, in <module>
   FluxCalibrateTask.parseAndRun()
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 603, in parseAndRun
   resultList = taskRunner.run(parsedCmd)
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
   resultList = list(mapFunc(self, targetList))
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 45, in _runPool
   return pool.map_async(function, iterable).get(timeout)
 File "/scratch/pprice/jenkins/weekly/2021-09-05/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 657, in get
   raise self._value
ValueError: operands could not be broadcast together with shapes (3,11376) (60,11376) (3,11376) 


 Comments   
Comment by price [ 08/Sep/21 ]

Simple fix.

Comment by hassan [ 08/Sep/21 ]

No comments to https://github.com/Subaru-PFS/drp_stella/pull/216 . Change looks fine.

Comment by price [ 08/Sep/21 ]

Merged.

Generated at Sat Feb 10 15:59:25 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.