[PIPE2D-830] Fix breakage in mergeArms

Fri, 7 May 2021 14:47:22 +0000

rhl writes:

I'm having trouble reducing SuNSS data with master. I built new fiberTraces using constructFiberProfiles.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS --id visit=46629..46649 --rerun rhl/sunss --batch-type none -c isr.doFlat=False isr.doDefect=False isr.doDark=False isr.doBrokenRedShutter=True isr.brokenRedShutter.useAnalytic=True isr.brokenRedShutter.checkParallelOverscan=True psfFwhm=3 forceFiberIds=True profiles.doBlindFind=False profiles.centroidRadius=3 profiles.profileRadius=3 rerunISR=False -j 20 (followed by ingest), but when I run reduceExposure and mergeArms I get this error. Looking at the pfsArm files in the r only shows around 5 spectra, possibly those which happen to have signal down the fibres.

 
$ reduceExposure.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS --rerun rhl/sunss -c doWriteCalexp=True isr.doFlat=False isr.doDefect=False isr.doDark=False repair.interp.modelPsf.defaultFwhm=2.25
$ mergeArms.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS --rerun rhl/sunss --id visit=46032 -c doBarycentricCorr=False doSubtractSky1d=False 
mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {512, 514, 3, 6, 518, 520, 9, 522, 12, 15, 527, 529, 18, 532, 21, 535, 24, 537, 27, 540, 30, 543, 32, 546, 35, 549, 38, 552, 41, 555, 46, 558, 49, 563, 52, 565, 55, 567, 58, 570, 572, 61, 574, 576, 66, 579, 69, 582, 72, 585, 75, 589, 78, 592, 81, 595, 84, 598, 87, 601, 90, 604, 95, 609, 98, 612, 101, 615, 104, 618, 107, 621, 110, 623, 113, 625, 627, 116, 630, 119, 632, 122, 634, 124, 636, 127, 639, 130, 642, 133, 645, 647, 649, 138, 141, 144, 147, 150, 153, 155, 156, 157, 158, 159, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 187, 189, 191, 193, 195, 197, 199, 201, 203, 205, 208, 210, 212, 215, 217, 219, 221, 223, 225, 227, 231, 233, 235, 238, 240, 242, 245, 247, 249, 251, 253, 257, 259, 261, 263, 266, 268, 270, 275, 278, 281, 284, 287, 290, 293, 296, 299, 302, 304, 306, 308, 311, 313, 315, 338, 341, 344, 347, 349, 351, 354, 357, 360, 363, 366, 369, 372, 375, 378, 381, 385, 387, 389, 391, 393, 395, 397, 399, 403, 406, 408, 410, 413, 415, 417, 419, 421, 423, 425, 429, 431, 433, 436, 438, 440, 443, 445, 447, 449, 451, 453, 455, 457, 459, 461, 466, 468, 474, 477, 479, 482, 485, 488, 491, 494, 497, 500, 502, 504, 506, 508, 510} (fixing) Only in armPsf: {192, 2, 339, 650}
mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {512, 514, 3, 6, 518, 520, 9, 522, 12, 15, 527, 529, 18, 532, 21, 535, 24, 537, 27, 540, 30, 543, 32, 546, 35, 549, 38, 552, 41, 555, 46, 558, 49, 563, 52, 565, 55, 567, 58, 570, 572, 61, 574, 576, 66, 579, 69, 582, 72, 585, 75, 589, 78, 592, 81, 595, 84, 598, 87, 601, 90, 604, 95, 609, 98, 612, 101, 615, 104, 618, 107, 621, 110, 623, 113, 625, 627, 116, 630, 119, 632, 122, 634, 124, 636, 127, 639, 130, 642, 133, 645, 647, 649, 138, 141, 144, 147, 150, 153, 155, 156, 157, 158, 159, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 187, 189, 191, 193, 195, 197, 199, 201, 203, 205, 208, 210, 212, 215, 217, 219, 221, 223, 225, 227, 231, 233, 235, 238, 240, 242, 245, 247, 249, 251, 253, 257, 259, 261, 263, 266, 268, 270, 275, 278, 281, 284, 287, 290, 293, 296, 299, 302, 304, 306, 308, 311, 313, 315, 338, 341, 344, 347, 349, 351, 354, 357, 360, 363, 366, 369, 372, 375, 378, 381, 385, 387, 389, 391, 393, 395, 397, 399, 403, 406, 408, 410, 413, 415, 417, 419, 421, 423, 425, 429, 431, 433, 436, 438, 440, 443, 445, 447, 449, 451, 453, 455, 457, 459, 461, 466, 468, 474, 477, 479, 482, 485, 488, 491, 494, 497, 500, 502, 504, 506, 508, 510} (fixing) Only in armPsf: {192, 2, 339, 650}
/home/rhl/PFS/drp/stella/python/pfs/drp/stella/datamodel/interpolate.py:61: RuntimeWarning: invalid value encountered in greater
  intIndex[(intIndex == index) & (index > 0)] -= 1  # Linear interpolation takes the index before
Traceback (most recent call last):
  File "/tigress/HSC/PFS/stack/current/stack/current/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
    result = self.runTask(task, dataRef, kwargs)
  File "/tigress/HSC/PFS/stack/current/stack/current/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
    return task.runDataRef(dataRef, **kwargs)
  File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 175, in runDataRef
    results = self.run(spectra, pfsConfig, lsfList)
  File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 125, in run
    lsfList = [self.mergeLsfs(ll, ss) for ll, ss in zip(lsfList, spectra)]
  File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 125, in <listcomp>
    lsfList = [self.mergeLsfs(ll, ss) for ll, ss in zip(lsfList, spectra)]
  File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 299, in mergeLsfs
    return {ff: coaddLsf([ww[ff] for ww in warpedLsfList]) for ff in fiberId}
  File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 299, in <dictcomp>
    return {ff: coaddLsf([ww[ff] for ww in warpedLsfList]) for ff in fiberId}
  File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 299, in <listcomp>
    return {ff: coaddLsf([ww[ff] for ww in warpedLsfList]) for ff in fiberId}
KeyError: 2

PFS-JIRA

[PIPE2D-830] Fix breakage in mergeArms