Uploaded image for project: 'DRP 2-D Pipeline'
  1. DRP 2-D Pipeline
  2. PIPE2D-892

Fix misshapen array in fluxCalibrate

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done (View Workflow)
    • Priority: Normal
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Sprint:
      2DDRP-2021 A 8

      Description

      The weekly failed:

      + fluxCalibrate.py /scratch/pprice/jenkins/weekly/2021-09-05/process --calib=/scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB --rerun=weekly/pipeline/brn/pipeline --doraise -j10 --id 'visit=47^48^49^53^54^57'
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      root INFO: Running: /scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/bin/fluxCalibrate.py /scratch/pprice/jenkins/weekly/2021-09-05/process --calib=/scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB --rerun=weekly/pipeline/brn/pipeline --doraise -j10 --id visit=47^48^49^53^54^57
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/registry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2021-09-05/process/CALIB/calibRegistry.sqlite3
      multiprocessing.pool.RemoteTraceback: 
      """
      Traceback (most recent call last):
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 121, in worker
         result = (True, func(*args, **kwds))
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
         return list(map(*args))
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 385, in __call__
         result = self.runTask(task, dataRef, kwargs)
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
         return task.runDataRef(dataRef, **kwargs)
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/python/pfs/drp/stella/fluxCalibrate.py", line 127, in runDataRef
         self.calculateCalibrations(mergedFluxCal, references)
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/python/pfs/drp/stella/fluxCalibrate.py", line 200, in calculateCalibrations
         merged.covar[0] += self.config.sysErr*merged.flux  # add systematic error
      ValueError: operands could not be broadcast together with shapes (3,11376) (60,11376) (3,11376) 
      """
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.36/bin/fluxCalibrate.py", line 3, in <module>
         FluxCalibrateTask.parseAndRun()
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 603, in parseAndRun
         resultList = taskRunner.run(parsedCmd)
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
         resultList = list(mapFunc(self, targetList))
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 45, in _runPool
         return pool.map_async(function, iterable).get(timeout)
       File "/scratch/pprice/jenkins/weekly/2021-09-05/build/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/multiprocessing/pool.py", line 657, in get
         raise self._value
      ValueError: operands could not be broadcast together with shapes (3,11376) (60,11376) (3,11376) 
      

        Attachments

          Activity

            People

            • Assignee:
              price price
              Reporter:
              price price
              Reviewers:
              hassan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: