Uploaded image for project: 'DRP 2-D Pipeline'
  1. DRP 2-D Pipeline
  2. PIPE2D-945

mergeArms fails on arc

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done (View Workflow)
    • Priority: Normal
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      ncaplar reports the following failure when running mergeArms.py on visit=67720, an arc:

      (lsst-scipipe) [ncaplar@tiger2-sumire ~]$ mergeArms.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/ --rerun ncaplar/PIPE2D-918_ApCorFalse_Nov11 --id visit=67720 -j 12
      CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
      CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
      root INFO: Running: /tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/bin/mergeArms.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/ --rerun ncaplar/PIPE2D-918_ApCorFalse_Nov11 --id visit=67720 -j 12
      CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
      CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
      CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
      mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {1, 515, 229, 137, 426, 651, 45, 336, 273, 560, 316, 471, 184, 92, 382, 607} (fixing)
      mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {1, 515, 229, 137, 426, 651, 45, 336, 273, 560, 316, 471, 184, 92, 382, 607} (fixing)
      multiprocessing.pool.RemoteTraceback: 
      """
      Traceback (most recent call last):
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/mergeArms.py", line 186, in runDataRef
          results = self.run(spectra, pfsConfig, lsfList)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/mergeArms.py", line 138, in run
          sky1d.append(self.skySubtraction(ss, pfsConfig))
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/mergeArms.py", line 409, in skySubtraction
          sky1d = self.fitSkyModel.run(skySpectra, skyConfig)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/fitFocalPlane.py", line 96, in run
          **fitParams)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/focalPlaneFunction.py", line 67, in fit
          raise RuntimeError("fiberId mismatch between spectra and pfsConfig")
      RuntimeError: fiberId mismatch between spectra and pfsConfig
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/multiprocessing/pool.py", line 121, in worker
          result = (True, func(*args, **kwds))
        File "/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
          return list(map(*args))
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 400, in __call__
          ", ".join(str(ref.dataId) for ref in dataRef), eName, e)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 400, in <genexpr>
          ", ".join(str(ref.dataId) for ref in dataRef), eName, e)
      AttributeError: 'list' object has no attribute 'dataId'
      """
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/bin/mergeArms.py", line 3, in <module>
          MergeArmsTask.parseAndRun()
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 603, in parseAndRun
          resultList = taskRunner.run(parsedCmd)
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
          resultList = list(mapFunc(self, targetList))
        File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 45, in _runPool
          return pool.map_async(function, iterable).get(timeout)
        File "/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/multiprocessing/pool.py", line 657, in get
          raise self._value
      AttributeError: 'list' object has no attribute 'dataId'
      

      The code is attempting to do sky subtraction, but there are no sky fibers. Fix the obscure error message.

        Attachments

          Activity

            People

            • Assignee:
              price price
              Reporter:
              price price
              Reviewers:
              hassan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: