[PIPE2D-945] mergeArms fails on arc

Thu, 11 Nov 2021 18:29:33 +0000
ncaplar reports the following failure when running mergeArms.py on visit=67720, an arc:
(lsst-scipipe) [ncaplar@tiger2-sumire ~]$ mergeArms.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/ --rerun ncaplar/PIPE2D-918_ApCorFalse_Nov11 --id visit=67720 -j 12
CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
root INFO: Running: /tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/bin/mergeArms.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/ --rerun ncaplar/PIPE2D-918_ApCorFalse_Nov11 --id visit=67720 -j 12
CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {1, 515, 229, 137, 426, 651, 45, 336, 273, 560, 316, 471, 184, 92, 382, 607} (fixing)
mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {1, 515, 229, 137, 426, 651, 45, 336, 273, 560, 316, 471, 184, 92, 382, 607} (fixing)
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
    result = self.runTask(task, dataRef, kwargs)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
    return task.runDataRef(dataRef, **kwargs)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/mergeArms.py", line 186, in runDataRef
    results = self.run(spectra, pfsConfig, lsfList)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/mergeArms.py", line 138, in run
    sky1d.append(self.skySubtraction(ss, pfsConfig))
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/mergeArms.py", line 409, in skySubtraction
    sky1d = self.fitSkyModel.run(skySpectra, skyConfig)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/fitFocalPlane.py", line 96, in run
    **fitParams)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/python/pfs/drp/stella/focalPlaneFunction.py", line 67, in fit
    raise RuntimeError("fiberId mismatch between spectra and pfsConfig")
RuntimeError: fiberId mismatch between spectra and pfsConfig

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 400, in __call__
    ", ".join(str(ref.dataId) for ref in dataRef), eName, e)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 400, in <genexpr>
    ", ".join(str(ref.dataId) for ref in dataRef), eName, e)
AttributeError: 'list' object has no attribute 'dataId'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45a/bin/mergeArms.py", line 3, in <module>
    MergeArmsTask.parseAndRun()
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 603, in parseAndRun
    resultList = taskRunner.run(parsedCmd)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
    resultList = list(mapFunc(self, targetList))
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 45, in _runPool
    return pool.map_async(function, iterable).get(timeout)
  File "/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AttributeError: 'list' object has no attribute 'dataId'
The code is attempting to do sky subtraction, but there are no sky fibers. Fix the obscure error message.
PFS-JIRA

[PIPE2D-945] mergeArms fails on arc
