Uploaded image for project: 'DRP 2-D Pipeline'
  1. DRP 2-D Pipeline
  2. PIPE2D-1209

Optimize reduceProfiles

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done (View Workflow)
    • Priority: Normal
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      reduceProfiles.py is peaking at 70% of memory use per spectrograph arm on machines at Hilo when running on the latest data. It's also taking a few hours per spectrograph arm. I'd like to improve both of these aspects.

      reduceProfiles.py /work/drp --calib=/work/drp/CALIB --rerun=price/20230428-profiles --doraise --id visit=91808^91837^91838^91840^91841^91843^91844^91907..91909^91936..91940^91942..91946^91948..91952 arm=r spectrograph=1 --normId visit=91907..91909 arm=r spectrograph=1 --no-versions -c reduceExposure.doAdjustDetectorMap=True profiles.profileRadius=10 profiles.profileOversample=3 profiles.profileSwath=2000 profiles.profileRejThresh=5 reduceExposure.isr.doFlat=False --no-versions -j 20

        Attachments

          Activity

            People

            • Assignee:
              price price
              Reporter:
              price price
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: