[PIPE2D-856] bootstrapDetectorMap fails with recent SuNSS m data Created: 18/Jun/21  Updated: 18/Jun/21  Resolved: 18/Jun/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: hassan Assignee: price
Resolution: Done Votes: 0
Labels: SuNSS
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 2
Sprint: 2DDRP-2021 A 6
Reviewers: hassan

 Description   

A recent problem occurred when trying to process SuNSS m-band data. Details below.

root INFO: Running: /tigress/hassans/software/drp_stella/bin/bootstrapDetectorMap.py /projects/HSC/PFS/Subaru/ --calib /projects/HSC/PFS/Subaru//CALIB-SuNSS-2021-06-16 --rerun hassan/mband5 --flatId visit=63050 arm=m --arcId visit=63106 arm=m -c isr.doDark=False
WARNING: You are using OpenBLAS with multiple threads (40), but have not
specified the number of threads using one of the OpenBLAS environment variables:
OPENBLAS_NUM_THREADS, GOTO_NUM_THREADS, OMP_NUM_THREADS.
This may indicate that you are unintentionally using multiple threads, which may
cause problems. WE HAVE THEREFORE DISABLED OpenBLAS THREADING. If you know
what you are doing and want threads enabled implicitly, set the environment
variable LSST_ALLOW_IMPLICIT_THREADS.
bootstrap.isr INFO: Performing ISR on sensor {'visit': 63050, 'arm': 'm', 'dateObs': '2021-06-11', 'site': 'S', 'category': 'A', 'spectrograph': 1, 'field': 'SEEING_TEST', 'ccd': 1, 'filter': 'm', 'expTime': 8.0, 'dataType': 'DOMEFLAT', 'taiObs': '2021-06-11T20:30:55.814', 'pfsDesignId': 3735928559, 'slitOffset': 0.0}
bootstrap.isr INFO: Converting exposure to floating point values
bootstrap.isr INFO: Assembling CCD from amplifiers
bootstrap.isr INFO: Applying bias correction.
bootstrap.isr INFO: Masking defects.
bootstrap.isr INFO: Masking NAN value pixels.
bootstrap.isr INFO: Widening saturation trails.
bootstrap.isr INFO: Interpolating masked pixels.
bootstrap.isr WARN: No rough magnitude zero point set for filter m
bootstrap.isr INFO: Setting rough magnitude zero point: 30.257725
bootstrap.profiles WARN: Failed to identify 221 fiberIds: [6, 9, 12, 15, 18, 21, 24, 27, 30, 32, 35, 38, 41, 49, 52, 55, 58, 61, 63, 66, 69, 72, 75, 78, 81, 84, 87, 90, 95, 98, 101, 104, 107, 110, 113, 116, 119, 122, 127, 130, 133, 141, 144, 147, 150, 153, 159, 182, 187, 189, 191, 193, 195, 197, 199, 201, 203, 205, 208, 210, 212, 215, 217, 219, 221, 223, 225, 227, 231, 233, 235, 240, 242, 245, 247, 249, 251, 253, 255, 257, 259, 261, 263, 266, 268, 270, 275, 278, 281, 284, 287, 290, 293, 296, 299, 302, 304, 306, 308, 311, 313, 338, 341, 344, 347, 349, 351, 354, 357, 360, 363, 366, 369, 372, 375, 378, 381, 385, 387, 389, 391, 393, 395, 397, 399, 401, 403, 406, 408, 410, 413, 415, 417, 419, 421, 423, 425, 429, 431, 433, 436, 438, 440, 443, 445, 447, 449, 451, 453, 457, 459, 461, 464, 466, 468, 474, 477, 479, 482, 485, 488, 491, 494, 497, 500, 502, 504, 506, 508, 510, 512, 514, 518, 520, 522, 525, 527, 529, 532, 535, 537, 540, 543, 546, 549, 552, 555, 558, 563, 565, 567, 570, 572, 574, 576, 579, 582, 585, 587, 589, 592, 595, 598, 601, 604, 612, 615, 618, 621, 623, 625, 627, 630, 632, 634, 636, 639, 642, 645, 647, 649]
bootstrap.profiles INFO: Profiled 254 fibers
bootstrap FATAL: Failed on dataId={'visit': 63050, 'arm': 'm', 'dateObs': '2021-06-11', 'site': 'S', 'category': 'A', 'spectrograph': 1, 'field': 'SEEING_TEST', 'ccd': 1, 'filter': 'm', 'expTime': 8.0, 'dataType': 'DOMEFLAT', 'taiObs': '2021-06-11T20:30:55.814', 'pfsDesignId': 3735928559, 'slitOffset': 0.0}: RuntimeError: Mismatch between number of traces (254) and number of fibers (251)
Traceback (most recent call last):
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
    result = self.runTask(task, dataRef, kwargs)
  File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
    return task.runDataRef(dataRef, **kwargs)
  File "/tigress/hassans/software/drp_stella/python/pfs/drp/stella/bootstrap.py", line 131, in runDataRef
    traces = self.traceFibers(flatRef, flatConfig)
  File "/tigress/hassans/software/drp_stella/python/pfs/drp/stella/bootstrap.py", line 172, in traceFibers
    (len(traces), len(indices)))
RuntimeError: Mismatch between number of traces (254) and number of fibers (251)


 Comments   
Comment by rhl [ 18/Jun/21 ]

My guess is that bootstrap hasn't been run since we labelled the three broken fibres in SuNSS (251 is the number of illuminated fibres;  there are 254 in the pfsConfig)

Comment by price [ 18/Jun/21 ]

As rhl suspected, the problem was that we're finding traces for broken fibers.

The fix requires PIPE2D-770 and PIPE2D-762.

Comment by hassan [ 18/Jun/21 ]

bootstrapDetectorMap runs to completion with these fixes.

Comment by hassan [ 18/Jun/21 ]

Updates made in pull request look fine to me.

Comment by price [ 18/Jun/21 ]

Merged.

Generated at Sat Feb 10 15:58:58 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.