[PIPE2D-1227] The weekly fails with w.2023.22

Thu, 6 Jul 2023 15:22:02 +0000
The weekly test for w.2022.22 failed at IPMU with the following error. I guess `W_SRH4` stuff was merged last week. Please confirm this and tell me if I did something wrong.
fiberProfiles INFO: Scatter processing
fiberProfiles INFO: Scatter processing
10365 INFO  2023-05-29T00:52:49.673+0900 pfs.fiberProfiles: Processing DataId(initialdata={'visit': 37, 'arm': 'r', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'A', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 1, 'filter': 'r', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set()) on 0ef8cba82b4f:10365
10366 INFO  2023-05-29T00:52:49.673+0900 pfs.fiberProfiles: Processing DataId(initialdata={'visit': 37, 'arm': 'n', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'B', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 2, 'filter': 'n', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set()) on 0ef8cba82b4f:10366
10365 INFO  2023-05-29T00:52:49.673+0900 pfs.fiberProfiles.isr: Performing ISR on sensor DataId(initialdata={'visit': 37, 'arm': 'r', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'A', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 1, 'filter': 'r', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set())
10366 INFO  2023-05-29T00:52:49.673+0900 pfs.fiberProfiles.isr: Performing ISR on sensor DataId(initialdata={'visit': 37, 'arm': 'n', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'B', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 2, 'filter': 'n', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set())
10364 INFO  2023-05-29T00:52:49.674+0900 pfs.fiberProfiles: Processing DataId(initialdata={'visit': 37, 'arm': 'b', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'A', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 0, 'filter': 'b', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set()) on 0ef8cba82b4f:10364
10364 INFO  2023-05-29T00:52:49.674+0900 pfs.fiberProfiles.isr: Performing ISR on sensor DataId(initialdata={'visit': 37, 'arm': 'b', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'A', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 0, 'filter': 'b', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set())
10366 WARN  2023-05-29T00:52:49.700+0900 pfs.fiberProfiles: Unable to process DataId(initialdata={'visit': 37, 'arm': 'n', 'dateObs': '2020-01-01', 'site': 'F', 'category': 'B', 'spectrograph': 1, 'field': 'FLAT_EVEN', 'ccd': 2, 'filter': 'n', 'expTime': 30.0, 'dataType': 'FLAT_EVEN', 'taiObs': '2020-01-01T00:37:00', 'pfsDesignId': 4, 'slitOffset': 0.0}, tag=set()): 'W_SRH4 not found'
/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py:304: FutureWarning: Gen2 Butler has been deprecated (PfsButler). It will be removed sometime after v23.0 but no earlier than the end of 2021.
  return super(Comm, self).bcast(value, root=root)
/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py:304: FutureWarning: Gen2 Butler has been deprecated (PfsMapper). It will be removed sometime after v23.0 but no earlier than the end of 2021.
  return super(Comm, self).bcast(value, root=root)
KeyError on 0ef8cba82b4f:10366 in run: 'W_SRH4 not found'
Traceback (most recent call last):
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 112, in wrapper
    return func(*args, **kwargs)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 1069, in run
    while not menu[command]():
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 239, in wrapper
    return func(*args, **kwargs)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 1087, in reduce
    result = self._processQueue(context, func, [(index, data)], *args, **kwargs)[0]
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 546, in _processQueue
    return self._reduceQueue(context, None, func, queue, *args, **kwargs)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 572, in _reduceQueue
    resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/ctrl_pool/g6ea9deef35+e480a1db32/python/lsst/ctrl/pool/pool.py", line 572, in <listcomp>
    resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_drivers/gef6abdcebb+582a64c339/python/lsst/pipe/drivers/constructCalibs.py", line 625, in process
    exposure = self.processSingle(sensorRef, **kwargs)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2023.22/python/pfs/drp/stella/constructSpectralCalibs.py", line 280, in processSingle
    exposure = super().processSingle(sensorRef)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_drivers/gef6abdcebb+582a64c339/python/lsst/pipe/drivers/constructCalibs.py", line 644, in processSingle
    return self.isr.runDataRef(dataRef).exposure
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/utils/g62e1f4f43a+9633a190c8/python/lsst/utils/timer.py", line 345, in timeMethod_wrapper
    res = func(self, *args, **keyArgs)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/obs_pfs/w.2023.22/python/lsst/obs/pfs/isrTask.py", line 356, in runDataRef
    ccdExposure = sensorRef.get(self.config.datasetType)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/daf_persistence/g6a31054a6e+4ae6138134/python/lsst/daf/persistence/butlerSubset.py", line 203, in get
    return self.butlerSubset.butler.get(datasetType, self.dataId, **rest)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/daf_persistence/g6a31054a6e+4ae6138134/python/lsst/daf/persistence/butler.py", line 1399, in get
    location = self._locate(datasetType, dataId, write=False)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/obs_pfs/w.2023.22/python/lsst/obs/pfs/dm35718.py", line 138, in _locate
    location.bypass = bypass()
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/obs_pfs/w.2023.22/python/lsst/obs/pfs/dm35718.py", line 174, in <lambda>
    return lambda: bypassFunc(location.datasetType, pythonType, location, dataId)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/obs_pfs/w.2023.22/python/lsst/obs/pfs/pfsMapper.py", line 295, in bypass_raw
    W_SRH4 = md["W_SRH4"]
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/daf_base/gcc072456be+b6f9ac6643/python/lsst/daf/base/propertyContainer/propertyContainerContinued.py", line 727, in __getitem__
    return self.getScalar(name)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/daf_base/gcc072456be+b6f9ac6643/python/lsst/daf/base/propertyContainer/propertyContainerContinued.py", line 878, in getScalar
    return _propertyContainerGet(self, name, returnStyle=ReturnStyle.SCALAR)
  File "/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/daf_base/gcc072456be+b6f9ac6643/python/lsst/daf/base/propertyContainer/propertyContainerContinued.py", line 215, in _propertyContainerGet
    raise KeyError(name + " not found")
KeyError: 'W_SRH4 not found'
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3
/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsButler). It will be removed sometime after v23.0 but no earlier than the end of 2021.
  namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)
/opt/pfs/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsMapper). It will be removed sometime after v23.0 but no earlier than the end of 2021.
  namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)
+ mkdir /arch1/ipmu/pipe_e2e/pipe2d_results/rerun/weekly/calib/brn/calibs_for_brn/fiberProfiles/FIBERPROFILES/COMBINED
mkdir: cannot create directory '/arch1/ipmu/pipe_e2e/pipe2d_results/rerun/weekly/calib/brn/calibs_for_brn/fiberProfiles/FIBERPROFILES/COMBINED': No such file or directory
PFS-JIRA

[PIPE2D-1227] The weekly fails with w.2023.22
