https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1209 DRP 2-D Pipeline <p><tt>reduceProfiles.py</tt> is peaking at 70% of memory use per spectrograph arm on machines at Hilo when running on the latest data. It's also taking a few hours per spectrograph arm. I'd like to improve both of these aspects.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> reduceProfiles.py /work/drp --calib=/work/drp/CALIB --rerun=price/20230428-profiles --doraise --id visit=91808^91837^91838^91840^91841^91843^91844^91907..91909^91936..91940^91942..91946^91948..91952 arm=r spectrograph=1 --normId visit=91907..91909 arm=r spectrograph=1 --no-versions -c reduceExposure.doAdjustDetectorMap=True profiles.profileRadius=10 profiles.profileOversample=3 profiles.profileSwath=2000 profiles.profileRejThresh=5 reduceExposure.isr.doFlat=False --no-versions -j 20 </pre> </div></div> PIPE2D-1209

Optimize reduceProfiles

Story Normal Done Done price price Fri, 28 Apr 2023 20:20:11 +0000 Tue, 16 May 2023 17:21:31 +0000 Tue, 16 May 2023 17:21:31 +0000 0 1 <p>Memory use was drastically reduced (70% --&gt; about 3%). Didn't make significant progress on speed, but I'm not too concerned about that right now since creating fiber profiles isn't a frequent operation.</p> <p>Merged to master.</p> Development Rank 0|zzswzz: