https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1107 DRP 2-D Pipeline <p>Presently the integration tests (and most likely future weekly builds) fail. A snippet of the log output of running the tests against master are shown below.</p> <p>I suspect that the problem is caused by the inclusion of <a href="https://pfspipe.ipmu.jp/jira/browse/PIPE2D-1094" title="Organise H4RG defects by logical serial number (snX) not spectrograph number (nY)" class="issue-link" data-issue-key="PIPE2D-1094"><del>PIPE2D-1094</del></a>. The new script introduced in that ticket should be used.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> 2114 + ingestCuratedCalibs.py /scratch/pprice/jenkins/integrationTest/461/INTEGRATION --calib /scratch/pprice/jenkins/integrationTest/461/INTEGRATION/CALIB /scratch/pprice/jenkin 2114 s/integrationTest/461/drp_pfs_data/curated/pfs/defects 2115 root INFO: Loading config overrride file <span class="code-quote">'/scratch/pprice/jenkins/integrationTest/461/obs_pfs/config/ingestCuratedCalibs.py'</span> 2116 lsst.CameraMapper INFO: Loading exposure registry from /scratch/pprice/jenkins/integrationTest/461/INTEGRATION/registry.sqlite3 2117 Traceback (most recent call last): 2118 File <span class="code-quote">"/projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_tasks/gb1d6de0934+6a8639994f/bin/ingestCuratedCalibs.py"</span>, line 3, in &lt;module&gt; 2119 IngestCuratedCalibsTask.parseAndRun() 2120 File <span class="code-quote">"/projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_tasks/gb1d6de0934+6a8639994f/python/lsst/pipe/tasks/ingest.py"</span>, line 431, in parseAnd 2120 Run 2121 task.run(args) 2122 File <span class="code-quote">"/projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_tasks/gb1d6de0934+6a8639994f/python/lsst/pipe/tasks/ingestCuratedCalibs.py"</span>, line 43, 2122 in run 2123 calibs, calib_type = read_all(args.root, camera) 2124 File <span class="code-quote">"/projects/HSC/PFS/stack/20220525/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_tasks/gb1d6de0934+6a8639994f/python/lsst/pipe/tasks/read_curated_calibs.py"</span>, line 139 2124 , in read_all 2125 raise RuntimeError(f<span class="code-quote">"Detector {chip_name} not known to supplied camera "</span> 2126 RuntimeError: Detector sn1 not known to supplied camera PFS (examples: AG5,AG4,r4,AG1,b4,b2,AG6,AG2,m2,r2) </pre> </div></div> PIPE2D-1107

Integration test failing at defect ingestion

Bug Blocker Done Done price hassan Wed, 2 Nov 2022 18:14:20 +0000 Tue, 7 Feb 2023 22:36:34 +0000 Fri, 4 Nov 2022 17:03:44 +0000 0 1 <p>Problem was indeed incorrect organization of H4RG defects. Updated integration test and weekly scripts to use new script to ingest defects.</p> <p>Integration test script still needs Gen3 section to be updated. Handing that to Paul.</p> <p>I changed a bunch of how this works.</p> <ul> <li>The script is now called <tt>makePfsDefects</tt>, and it lives in drp&#95;pfs&#95;data.</li> <li>The <tt>snX</tt> directories with the NIR defects are now in <tt>drp&#95;pfs&#95;data/nirDefects</tt>.</li> <li><tt>makePfsDefects</tt> builds the defects in <tt>drp&#95;pfs&#95;data/curated/pfs/defects</tt>, because that's what Gen3 requires.</li> <li><tt>makePfsDefects</tt> does not ingest the defects, because that's done differently for Gen2 and Gen3.</li> </ul> <p>Reviewed. All changes look fine, but shouldn't <tt>process_weekly.sh</tt> also be updated to use <tt>makePfsDefects</tt>? </p> <p>Added the change to the weekly, and merged.</p> Relates PIPE2D-1094 Development Rank 0|02qph7:0f Reviewers hassan Sprint 2DDRP-2023 A Story Points 2.0