https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-224 DRP 2-D Pipeline <p>The new support for proper unit exit codes (LSST's DM-4141) requires some boiler plate added to TaskRunner; please add it.</p> PIPE2D-224

Add support for DM-4141 to reduceArc*.py

Story Normal Done Done rhl rhl Sat, 22 Jul 2017 13:13:23 +0000 Tue, 25 Jul 2017 20:15:56 +0000 Tue, 25 Jul 2017 20:15:56 +0000 0 2 <p>I <b>think</b> you could get rid of <tt>ReduceArcTaskRunner.&#95;&#95;call&#95;&#95;</tt> completely (along with the associated maintenance) if you changed the <tt>getTargetList</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">class </span>ReduceArcTaskRunner(TaskRunner): <span class="code-quote">"""Get parsed values into the ReduceArcTask.run"</span>"" @staticmethod def getTargetList(parsedCmd, **kwargs): <span class="code-keyword">return</span> [(parsedCmd.id.refList, dict(butler=parsedCmd.butler, wLenFile=parsedCmd.wLenFile, lineList=parsedCmd.lineList))] </pre> </div></div> <p>That fails with </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Traceback (most recent call last): File <span class="code-quote">"bin/reduceArcRefSpec.py"</span>, line 4, in &lt;module&gt; ReduceArcRefSpecTask.parseAndRun() File <span class="code-quote">"/Users/rhl/LSST/pipe/base/python/lsst/pipe/base/cmdLineTask.py"</span>, line 532, in parseAndRun resultList = taskRunner.run(parsedCmd) File <span class="code-quote">"/Users/rhl/LSST/pipe/base/python/lsst/pipe/base/cmdLineTask.py"</span>, line 227, in run resultList = list(mapFunc(self, targetList)) File <span class="code-quote">"/Users/rhl/LSST/pipe/base/python/lsst/pipe/base/cmdLineTask.py"</span>, line 368, in __call__ dataRef, kwargs = args ValueError: too many values to unpack </pre> </div></div> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> (Pdb) w /Users/rhl/PFS/drp/stella/bin/reduceArcRefSpec.py(4)&lt;module&gt;() -&gt; ReduceArcRefSpecTask.parseAndRun() /Users/rhl/LSST/pipe/base/python/lsst/pipe/base/cmdLineTask.py(532)parseAndRun() -&gt; resultList = taskRunner.run(parsedCmd) /Users/rhl/LSST/pipe/base/python/lsst/pipe/base/cmdLineTask.py(227)run() -&gt; resultList = list(mapFunc(self, targetList)) &gt; /Users/rhl/LSST/pipe/base/python/lsst/pipe/base/cmdLineTask.py(368)__call__() -&gt; dataRef, kwargs = args (Pdb) p type(args) &lt;type <span class="code-quote">'dict'</span>&gt; (Pdb) p args.keys() [<span class="code-quote">'butler'</span>, <span class="code-quote">'lineList'</span>, <span class="code-quote">'expRefList'</span>, <span class="code-quote">'refSpec'</span>] </pre> </div></div> <p>If <tt>type(args)</tt> is <tt>dict</tt> then I don't think you can be using the code I provided, where <tt>type(args)</tt> would be <tt>tuple</tt>?</p> <p>Sorry, you're right (I cut and pasted something foolish from pdb). But there is still a problem; <tt>dataRef</tt> is a list, and it's passed to <tt>task.writeMetadata</tt>.</p> <p>You could add to <tt>ReduceArcTask</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> def _getMetadataName(self): <span class="code-keyword">return</span> None </pre> </div></div> <p>Why does <tt>reduceArc*.py</tt> need a list of input dataRefs?</p> <p>Paul and I discussed this.</p> <p><tt>reduceArc.py</tt> was designed to add sets of exposures together to improve the S/N and then write a calibration product for later reduction. It currently processes individual images, but I'd rather not address this problem with this ticket.</p> <p>Paul's proposal to simply turn off metadata is a workaround, but as the metadata dumping doesn't work sensibly for <b>any</b> <tt>pipe_driver</tt> scripts which process multiple {{dataRef}}s it's acceptable here.</p> <p>rebased, merged, pushed</p> Development Rank 0|ii02i7: Reviewers price