https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1049 DRP 2-D Pipeline <p>The weekly deadlocked. This appears to be fairly reproducible with <tt>w.2022.22</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> (lsst-scipipe-3.0.0) pprice@tiger2-sumire:~ $ reduceArc.py /scratch/pprice/jenkins/weekly/2022-05-29/process --calib=/scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB --rerun=test --doraise --id visit=39..45:2 --config fitDetectorMap.doSlitOffsets=True reduceExposure.readLineList.exclusionRadius=0.3 fitDetectorMap.minSignalToNoise=20 -j 10 root INFO: Loading config overrride file <span class="code-quote">'/projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/obs_pfs/w.2022.22/config/reduceArc.py'</span> /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (Butler). It will be removed sometime after v23.0 but no earlier than the end of 2021. namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs) /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsMapper). It will be removed sometime after v23.0 but no earlier than the end of 2021. namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs) lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 root INFO: Running: /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/bin/reduceArc.py /scratch/pprice/jenkins/weekly/2022-05-29/process --calib=/scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB --rerun=test --doraise --id visit=39..45:2 --config fitDetectorMap.doSlitOffsets=True reduceExposure.readLineList.exclusionRadius=0.3 fitDetectorMap.minSignalToNoise=20 -j 10 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 45, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:45:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 41, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 5.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:41:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 39, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 2.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:39:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 39, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 2.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:39:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 43, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:43:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 39, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 2.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:39:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 43, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:43:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 41, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 5.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:41:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 43, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:43:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 41, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 5.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:41:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 41, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 5.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:41:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 45, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:45:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 41, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 5.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:41:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 41, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 5.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:41:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 43, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:43:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 43, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:43:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 43, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:43:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 39, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 2.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:39:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 39, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 0, <span class="code-quote">'filter'</span>: <span class="code-quote">'b'</span>, <span class="code-quote">'expTime'</span>: 2.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:39:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 39, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 2.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:39:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Set 29 BAD pixels to 0.616691. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Set 38 BAD pixels to 0.783048. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Set 32 BAD pixels to 0.449544. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.repair INFO: Identified 3151 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 6475 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 1878 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 1847 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 807 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 46 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 32 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 33 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 37 cosmic rays. reduceArc.reduceExposure.repair INFO: Identified 15888 cosmic rays. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'HgI'</span>, <span class="code-quote">'ArI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'NeI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'HgI'</span>, <span class="code-quote">'ArI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'HgI'</span>, <span class="code-quote">'ArI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'XeI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'XeI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'XeI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'KrI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'NeI'</span>}. reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'NeI'</span>}. reduceArc.reduceExposure.centroidLines INFO: Measured 43442 line centroids reduceArc.reduceExposure.centroidLines INFO: Measured 54434 line centroids reduceArc.reduceExposure.centroidLines INFO: Measured 88800 line centroids reduceArc.reduceExposure.centroidLines INFO: Measured 79918 line centroids reduceArc.reduceExposure.centroidLines INFO: Measured 145968 line centroids reduceArc.reduceExposure.centroidLines INFO: Measured 94348 line centroids reduceArc.reduceExposure.photometerLines INFO: Photometered 43442 lines reduceArc.reduceExposure.centroidLines INFO: Measured 168085 line centroids reduceArc.reduceExposure.photometerLines INFO: Photometered 54434 lines /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 45, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 1, <span class="code-quote">'filter'</span>: <span class="code-quote">'r'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:45:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /scratch/pprice/jenkins/weekly/2022-05-29/process/CALIB/calibRegistry.sqlite3 reduceArc.reduceExposure INFO: Processing [{<span class="code-quote">'visit'</span>: 45, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:45:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0}] reduceArc.reduceExposure.isr INFO: Performing ISR on sensor {<span class="code-quote">'visit'</span>: 45, <span class="code-quote">'dateObs'</span>: <span class="code-quote">'2020-01-01'</span>, <span class="code-quote">'site'</span>: <span class="code-quote">'F'</span>, <span class="code-quote">'category'</span>: <span class="code-quote">'A'</span>, <span class="code-quote">'arm'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'spectrograph'</span>: 1, <span class="code-quote">'field'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'ccd'</span>: 2, <span class="code-quote">'filter'</span>: <span class="code-quote">'n'</span>, <span class="code-quote">'expTime'</span>: 1.0, <span class="code-quote">'dataType'</span>: <span class="code-quote">'ARC'</span>, <span class="code-quote">'taiObs'</span>: <span class="code-quote">'2020-01-01T00:45:00'</span>, <span class="code-quote">'pfsDesignId'</span>: 1, <span class="code-quote">'slitOffset'</span>: 0.0} reduceArc.reduceExposure.centroidLines INFO: Measured 171200 line centroids lsst.CameraMapper INFO: darkTime is NaN/Inf; using exposureTime reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.centroidLines INFO: Measured 187888 line centroids reduceArc.reduceExposure.isr WARN: filterLabel is None. Setting physicalFilter to <span class="code-quote">"Unknown"</span>. reduceArc.reduceExposure.isr INFO: Converting exposure to floating point values. reduceArc.reduceExposure.isr INFO: Assembling CCD from amplifiers. reduceArc.reduceExposure.isr INFO: Applying bias correction. reduceArc.reduceExposure.isr INFO: Masking defects. reduceArc.reduceExposure.isr INFO: Masking non-finite (NAN, inf) value pixels. reduceArc.reduceExposure.isr INFO: Widening saturation trails. reduceArc.reduceExposure.isr INFO: Applying dark correction. reduceArc.reduceExposure.isr INFO: Applying flat correction. reduceArc.reduceExposure.isr INFO: Interpolating masked pixels. reduceArc.reduceExposure.photometerLines INFO: Photometered 88800 lines reduceArc.reduceExposure.photometerLines INFO: Photometered 79918 lines reduceArc.reduceExposure.repair INFO: Identified 33 cosmic rays. /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.readLineList INFO: Filtered line lists, keeping species {<span class="code-quote">'KrI'</span>}. /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.photometerLines INFO: Photometered 94348 lines /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.photometerLines INFO: Photometered 145968 lines reduceArc.reduceExposure.centroidLines INFO: Measured 48456 line centroids /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.photometerLines INFO: Photometered 168085 lines /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.photometerLines INFO: Photometered 171200 lines reduceArc.reduceExposure.photometerLines INFO: Photometered 48456 lines reduceArc.reduceExposure.photometerLines INFO: Photometered 187888 lines /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.centroidLines INFO: Measured 336600 line centroids /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> reduceArc.reduceExposure.photometerLines INFO: Photometered 336600 lines /projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2022.22/python/pfs/drp/stella/pickleUtils.py:9: FutureWarning: Call to deprecated method getExposureId. (Replaced by VisitInfo.id <span class="code-keyword">for</span> full focal plane identifiers and by ExposureInfo.id <span class="code-keyword">for</span> detector-level identifiers. Will be removed after v25.) -- Deprecated since version v24.0. tuple(getattr(info, <span class="code-quote">"get"</span> + prop)() <span class="code-keyword">for</span> </pre> </div></div> PIPE2D-1049

Deadlock in weekly processing

Story Normal Done Done price price Wed, 1 Jun 2022 21:17:38 +0000 Fri, 10 Jun 2022 15:50:16 +0000 Fri, 10 Jun 2022 15:50:16 +0000 0 3 <p>At least some of the deadlocks are coming from the <tt>threading.RLock</tt> in <tt>astropy.utils.decorators.lazyproperty</tt>. Removing that allows the operation to work more often.</p> <p>I believe the deadlocks are occurring because the lock is being created and then the process is forked (as part of <tt>multiprocessing</tt>), resulting in a process that has a lock that's meant for a different process. I think the locking should be removed from <tt>lazyproperty</tt> (and any locking implemented at a higher level). An alternative would be to store the locks by PID, but then I think there's a race condition on creating the lock (we'd need a lock on the creation of the lock...).</p> <p>Do we need <tt>astropy.utils.decorators.lazyproperty</tt>? It looks cute, but dangerous. How hard is it to implement something similar for the cases that the performance is important?</p> <p>The problem is that <tt>astropy.io.fits</tt> uses it.</p> <p>Bruce Merry (author of the astropy thread locking code) has pointed out that there are intrinsic problems with multiprocessing using the "fork" start method and <tt>maxtasksperchild=1</tt>, and I believe we're running afoul of those for a reason I haven't been able to discern. I've not been able to get the "forkserver" or "spawn" start methods to work reliably in order to work around this problem that way. But as far as we know, this is a Gen2-only problem, and Gen2 is dying, so I'm putting in the only workaround that I've been able to get to work, which is stripping the locks out of <tt>lazyproperty</tt>. We should be able to remove it once we burn Gen2 with fire.</p> <p>Workaround looks fine to me.</p> <p>Merged.</p> Development Rank 0|02qpjd:00xznup2c6 Reviewers hassan Sprint 2DDRP-2022 E