Uploaded image for project: 'DRP 2-D Pipeline'
  1. DRP 2-D Pipeline
  2. PIPE2D-830

Fix breakage in mergeArms

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done (View Workflow)
    • Priority: Normal
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Sprint:
      2DDRP-2021 A 4, 2DDRP-2021 A5

      Description

      rhl writes:

      I'm having trouble reducing SuNSS data with master. I built new fiberTraces using constructFiberProfiles.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS --id visit=46629..46649 --rerun rhl/sunss --batch-type none -c isr.doFlat=False isr.doDefect=False isr.doDark=False isr.doBrokenRedShutter=True isr.brokenRedShutter.useAnalytic=True isr.brokenRedShutter.checkParallelOverscan=True psfFwhm=3 forceFiberIds=True profiles.doBlindFind=False profiles.centroidRadius=3 profiles.profileRadius=3 rerunISR=False -j 20 (followed by ingest), but when I run reduceExposure and mergeArms I get this error. Looking at the pfsArm files in the r only shows around 5 spectra, possibly those which happen to have signal down the fibres.

       
      $ reduceExposure.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS --rerun rhl/sunss -c doWriteCalexp=True isr.doFlat=False isr.doDefect=False isr.doDark=False repair.interp.modelPsf.defaultFwhm=2.25
      $ mergeArms.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS --rerun rhl/sunss --id visit=46032 -c doBarycentricCorr=False doSubtractSky1d=False 
      mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {512, 514, 3, 6, 518, 520, 9, 522, 12, 15, 527, 529, 18, 532, 21, 535, 24, 537, 27, 540, 30, 543, 32, 546, 35, 549, 38, 552, 41, 555, 46, 558, 49, 563, 52, 565, 55, 567, 58, 570, 572, 61, 574, 576, 66, 579, 69, 582, 72, 585, 75, 589, 78, 592, 81, 595, 84, 598, 87, 601, 90, 604, 95, 609, 98, 612, 101, 615, 104, 618, 107, 621, 110, 623, 113, 625, 627, 116, 630, 119, 632, 122, 634, 124, 636, 127, 639, 130, 642, 133, 645, 647, 649, 138, 141, 144, 147, 150, 153, 155, 156, 157, 158, 159, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 187, 189, 191, 193, 195, 197, 199, 201, 203, 205, 208, 210, 212, 215, 217, 219, 221, 223, 225, 227, 231, 233, 235, 238, 240, 242, 245, 247, 249, 251, 253, 257, 259, 261, 263, 266, 268, 270, 275, 278, 281, 284, 287, 290, 293, 296, 299, 302, 304, 306, 308, 311, 313, 315, 338, 341, 344, 347, 349, 351, 354, 357, 360, 363, 366, 369, 372, 375, 378, 381, 385, 387, 389, 391, 393, 395, 397, 399, 403, 406, 408, 410, 413, 415, 417, 419, 421, 423, 425, 429, 431, 433, 436, 438, 440, 443, 445, 447, 449, 451, 453, 455, 457, 459, 461, 466, 468, 474, 477, 479, 482, 485, 488, 491, 494, 497, 500, 502, 504, 506, 508, 510} (fixing) Only in armPsf: {192, 2, 339, 650}
      mergeArms WARN: Set of fiberIds of LSFs != fiberIds for spectra:  Only in fiberId: {512, 514, 3, 6, 518, 520, 9, 522, 12, 15, 527, 529, 18, 532, 21, 535, 24, 537, 27, 540, 30, 543, 32, 546, 35, 549, 38, 552, 41, 555, 46, 558, 49, 563, 52, 565, 55, 567, 58, 570, 572, 61, 574, 576, 66, 579, 69, 582, 72, 585, 75, 589, 78, 592, 81, 595, 84, 598, 87, 601, 90, 604, 95, 609, 98, 612, 101, 615, 104, 618, 107, 621, 110, 623, 113, 625, 627, 116, 630, 119, 632, 122, 634, 124, 636, 127, 639, 130, 642, 133, 645, 647, 649, 138, 141, 144, 147, 150, 153, 155, 156, 157, 158, 159, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 187, 189, 191, 193, 195, 197, 199, 201, 203, 205, 208, 210, 212, 215, 217, 219, 221, 223, 225, 227, 231, 233, 235, 238, 240, 242, 245, 247, 249, 251, 253, 257, 259, 261, 263, 266, 268, 270, 275, 278, 281, 284, 287, 290, 293, 296, 299, 302, 304, 306, 308, 311, 313, 315, 338, 341, 344, 347, 349, 351, 354, 357, 360, 363, 366, 369, 372, 375, 378, 381, 385, 387, 389, 391, 393, 395, 397, 399, 403, 406, 408, 410, 413, 415, 417, 419, 421, 423, 425, 429, 431, 433, 436, 438, 440, 443, 445, 447, 449, 451, 453, 455, 457, 459, 461, 466, 468, 474, 477, 479, 482, 485, 488, 491, 494, 497, 500, 502, 504, 506, 508, 510} (fixing) Only in armPsf: {192, 2, 339, 650}
      /home/rhl/PFS/drp/stella/python/pfs/drp/stella/datamodel/interpolate.py:61: RuntimeWarning: invalid value encountered in greater
        intIndex[(intIndex == index) & (index > 0)] -= 1  # Linear interpolation takes the index before
      Traceback (most recent call last):
        File "/tigress/HSC/PFS/stack/current/stack/current/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/tigress/HSC/PFS/stack/current/stack/current/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 175, in runDataRef
          results = self.run(spectra, pfsConfig, lsfList)
        File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 125, in run
          lsfList = [self.mergeLsfs(ll, ss) for ll, ss in zip(lsfList, spectra)]
        File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 125, in <listcomp>
          lsfList = [self.mergeLsfs(ll, ss) for ll, ss in zip(lsfList, spectra)]
        File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 299, in mergeLsfs
          return {ff: coaddLsf([ww[ff] for ww in warpedLsfList]) for ff in fiberId}
        File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 299, in <dictcomp>
          return {ff: coaddLsf([ww[ff] for ww in warpedLsfList]) for ff in fiberId}
        File "/home/rhl/PFS/drp/stella/python/pfs/drp/stella/mergeArms.py", line 299, in <listcomp>
          return {ff: coaddLsf([ww[ff] for ww in warpedLsfList]) for ff in fiberId}
      KeyError: 2
      

        Attachments

          Activity

            People

            • Assignee:
              price price
              Reporter:
              price price
              Reviewers:
              hassan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: