[PIPE2D-524] Fix trace finding Created: 17/Mar/20  Updated: 11/Feb/22  Resolved: 17/Mar/20

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: price Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: 2DDRP-2021 A
Reviewers: hassan

 Description   

In building calibs for testing SIM2D-119, I hit a MemoryError: std::bad_array_new_length in constructFlat, because we failed to find any traces for one of the combined dithers.

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/sim2d-119 $ makeDummyCableBDesign.py red1 green red4 red8
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/sim2d-119 $ mkdir DATA
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/sim2d-119 $ mkdir DATA/CALIB
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/sim2d-119 $ echo lsst.obs.pfs.PfsMapper > DATA/_mapper
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/sim2d-119 $ ingestPfsImages.py DATA '/tigress/HSC/PFS/LAM/raw/2019-07-2[56]/*.fits' --pfsConfigDir .
(lsst-scipipe) pprice@tiger2-sumire:/scratch/pprice/sim2d-119 $ ingestCalibs.py DATA --calib DATA/CALIB $DRP_PFS_DATA_DIR/detectorMap/detectorMap-2019Jul-?1.fits --validity=1000 --mode=copy
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/sim2d-119 $ pfs_build_calibs.sh -r calibs -C DATA/CALIB -c 4 -n -b visit=21077..21091 -d visit=21092..21121 -f visit=21127..21255 -F visit=21122 -a visit=21460 DATA

(lsst-scipipe) pprice@tiger2-sumire:/scratch/pprice/sim2d-119 $ constructFiberFlat.py DATA --calib DATA/CALIB --rerun calibs/flat --id visit=21127..21255 --batch-type=smp --cores 10 --doraise

fiberFlat INFO: Combined 3 images for dither -0.0407
fiberFlat INFO: 0 FiberTraces found for dither -0.0407

MemoryError on tiger2-sumire.princeton.edu:59221 in run: std::bad_array_new_length
Traceback (most recent call last):
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 113, in wrapper
    return func(*args, **kwargs)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 1070, in run
    while not menu[command]():
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 240, in wrapper
    return func(*args, **kwargs)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 1088, in reduce
    result = self._processQueue(context, func, [(index, data)], *args, **kwargs)[0]
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 547, in _processQueue
    return self._reduceQueue(context, None, func, queue, *args, **kwargs)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 573, in _reduceQueue
    resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/ctrl_pool/18.1.0/python/lsst/ctrl/pool/pool.py", line 573, in <listcomp>
    resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/5.1.0-71-ga2d7c8b/python/pfs/drp/stella/constructFiberFlatTask.py", line 102, in combine
    spectra = traces.extractSpectra(image, maskVal)
MemoryError: std::bad_array_new_length
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2


 Comments   
Comment by price [ 17/Mar/20 ]

I think this is an important fix that we should merge soon.

Comment by price [ 17/Mar/20 ]

Thanks Hassan!

Merged to master.

Generated at Sat Feb 10 15:54:25 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.