PFS-JIRA https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 [PIPE2D-709] Generate recent bootstrapped detectorMap for Subaru https://pfspipe.ipmu.jp/jira/browse/PIPE2D-709 DRP 2-D Pipeline <p>To aid reduction of newly-acquired SuNSS data, generate a new bootstrapped detectorMap for Subaru SM1 from Dummy Cable B data just prior to the installation of SuNSS.</p> PIPE2D-709 Generate recent bootstrapped detectorMap for Subaru Story Normal Done Done price price Fri, 12 Feb 2021 16:42:41 +0000 Mon, 15 Feb 2021 23:30:11 +0000 Mon, 15 Feb 2021 22:05:12 +0000 0 1 <p>Delivered on Friday.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Found a suitable arc (45744) and flat (45742 <span class="code-keyword">for</span> r; 45739 <span class="code-keyword">for</span> b). Biases: 45677..45691 Darks: 45692..45706 (lsst-scipipe) pprice@tiger2-sumire:/scratch/pprice/pipe2d-709 $ generateCommands.py /projects/HSC/PFS/Subaru/ --calib /tigress/HSC/PFS/Subaru/CALIB-price --rerun price/pipe2d-709 --blocks=pipe2d_709 -j 20 pipe2d-709.yaml pipe2d-709.sh Need init block (PIPE2D-712). Bootstrap isn't generated correctly (PIPE2D-713). Let's run the commands individually ourselves. constructPfsBias.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/bias --doraise --batch-type=smp --cores=20 --id visit=45677..45691 ingestPfsCalibs.py /projects/HSC/PFS/Subaru --output=/tigress/HSC/PFS/Subaru/CALIB-price --validity=1800 --doraise --mode=copy -- /projects/HSC/PFS/Subaru/rerun/price/pipe2d-709/bias/BIAS/*.fits constructPfsDark.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/dark --doraise --batch-type=smp --cores=20 --id visit=45692..45706 ingestPfsCalibs.py /projects/HSC/PFS/Subaru --output=/tigress/HSC/PFS/Subaru/CALIB-price --validity=1800 --doraise --mode=copy -- /projects/HSC/PFS/Subaru/rerun/price/pipe2d-709/dark/DARK/*.fits (lsst-scipipe) pprice@tiger2-sumire:/scratch/pprice/pipe2d-709 $ bootstrapDetectorMap.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun price/pipe2d-709/bootstrap --flatId visit=45742 --arcId visit=45744 arm=r -c spatialOrder=1 spectralOrder=1 isr.doFlat=False bootstrap INFO: Found 452 lines in 10 traces bootstrap INFO: Matched 376 lines bootstrap INFO: Median difference from detectorMap: -0.026071,-0.671835 pixels bootstrap INFO: Fit 214/226 points, rms: x=0.698144 y=0.155658 total=0.452632 pixels bootstrap INFO: Median difference from detectorMap: -0.145875,-0.682627 pixels bootstrap INFO: Fit 107/150 points, rms: x=0.285516 y=0.061439 total=0.159123 pixels (lsst-scipipe) pprice@tiger2-sumire:/scratch/pprice/pipe2d-709 $ bootstrapDetectorMap.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun price/pipe2d-709/bootstrap --flatId visit=45739 --arcId visit=45744 arm=b -c spatialOrder=1 spectralOrder=1 isr.doFlat=False bootstrap INFO: Found 184 lines in 10 traces bootstrap INFO: Matched 174 lines bootstrap INFO: Median difference from detectorMap: 0.042010,-0.560648 pixels bootstrap INFO: Fit 96/104 points, rms: x=0.716973 y=0.073478 total=0.464745 pixels bootstrap INFO: Median difference from detectorMap: -0.176564,-0.532863 pixels bootstrap INFO: Fit 64/70 points, rms: x=0.559094 y=0.066189 total=0.324827 pixels ingestPfsCalibs.py /projects/HSC/PFS/Subaru --output=/tigress/HSC/PFS/Subaru/CALIB-price --validity=1800 --doraise --mode=copy --config clobber=True -- /projects/HSC/PFS/Subaru/rerun/price/pipe2d-709/bootstrap/DETECTORMAP/*.fits constructFiberProfiles.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/profiles --doraise --batch-type=smp --cores=20 --id visit=45739^45742 -c isr.doFlat=False ingestPfsCalibs.py /projects/HSC/PFS/Subaru --output=/tigress/HSC/PFS/Subaru/CALIB-price --validity=1800 --doraise --mode=copy --config clobber=True -- /projects/HSC/PFS/Subaru/rerun/price/pipe2d-709/profiles/FIBERPROFILES/*.fits reduceArc.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/detectorMap --doraise --id visit=45743..45746 arm=b -c fitDetectorMap.order=3 reduceExposure.isr.doFlat=False reduceArc.fitDetectorMap INFO: Final fit: chi2=1035.879258 xRMS=0.095182 yRMS=0.062718 (0.004274 nm, 2.476726 km/s) from 209/418 lines reduceArc.fitDetectorMap INFO: Fit quality from reserved lines: chi2=27357.479858 xRMS=0.534432 yRMS=0.195452 (0.013320 nm, 7.718392 km/s) from 47 lines (10.1%) reduceArc.fitDetectorMap INFO: Softening errors by 0.206139 pixels (0.014048 nm, 8.140385 km/s) to yield chi^2/dof=1 reduceArc.fitDetectorMap INFO: Softened fit: chi2=213.823215 xRMS=0.194306 yRMS=0.228703 (0.015586 nm, 9.031459 km/s) from 209/465 lines reduceArc.fitDetectorMap INFO: Softened fit quality from reserved lines: chi2=773.751027 xRMS=0.360469 yRMS=0.202646 (0.013810 nm, 8.002454 km/s) from 47 lines (10.1%) Not happy with that result at all, except that I actually got a result. reduceArc.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/detectorMap --doraise --id visit=45743..45746 arm=r -c fitDetectorMap.order=3 reduceExposure.isr.doFlat=False reduceArc.fitDetectorMap INFO: Final fit: chi2=1995.563540 xRMS=0.118448 yRMS=0.086120 (0.007385 nm, 2.757787 km/s) from 269/1315 lines reduceArc.fitDetectorMap INFO: Fit quality from reserved lines: chi2=620447.612288 xRMS=0.867036 yRMS=0.324389 (0.027817 nm, 10.387740 km/s) from 146 lines (10.0%) reduceArc.fitDetectorMap INFO: Softening errors by 0.305432 pixels (0.026191 nm, 9.780684 km/s) to yield chi^2/dof=1 reduceArc.fitDetectorMap INFO: Softened fit: chi2=323.601180 xRMS=0.293803 yRMS=0.273889 (0.023486 nm, 8.770592 km/s) from 269/1461 lines reduceArc.fitDetectorMap INFO: Softened fit quality from reserved lines: chi2=1409.068308 xRMS=0.557625 yRMS=0.338038 (0.028987 nm, 10.824803 km/s) from 146 lines (10.0%) That's even worse! With PIPE2D-660 (DifferentialDetectorMap), we <span class="code-keyword">do</span> MUCH better! Merged PIPE2D-660. reduceArc.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/detectorMap --doraise --id visit=45743..45746 arm=b -c reduceExposure.isr.doFlat=False reduceArc.fitDetectorMap INFO: Final fit: chi2=673.760298 dof=706 xRMS=0.019120 yRMS=0.025782 (0.001751 nm, 1.014676 km/s) from 381/419 lines reduceArc.fitDetectorMap INFO: Fit quality from reserved lines: chi2=113.399519 xRMS=0.084767 yRMS=0.127548 (0.008663 nm, 5.019814 km/s) from 47 lines (10.1%) reduceArc.fitDetectorMap INFO: Softening errors by 0.008998 pixels (0.000611 nm, 0.354139 km/s) to yield chi^2/dof=1 reduceArc.fitDetectorMap INFO: Softened fit: chi2=705.803279 dof=706 xRMS=0.018530 yRMS=0.024717 (0.001679 nm, 0.972766 km/s) from 381/466 lines reduceArc.fitDetectorMap INFO: Softened fit quality from reserved lines: chi2=120.593685 xRMS=0.085329 yRMS=0.127512 (0.008661 nm, 5.018415 km/s) from 47 lines (10.1%) reduceArc.py /projects/HSC/PFS/Subaru --calib=/tigress/HSC/PFS/Subaru/CALIB-price --rerun=price/pipe2d-709/detectorMap --doraise --id visit=45743..45746 arm=r -c reduceExposure.isr.doFlat=False reduceArc.fitDetectorMap INFO: Final fit: chi2=2689.543040 dof=1976 xRMS=0.025168 yRMS=0.026273 (0.002255 nm, 0.841970 km/s) from 1016/1329 lines reduceArc.fitDetectorMap INFO: Fit quality from reserved lines: chi2=20502.149823 xRMS=0.055518 yRMS=0.048970 (0.004203 nm, 1.569353 km/s) from 148 lines (10.0%) reduceArc.fitDetectorMap INFO: Softening errors by 0.020690 pixels (0.001776 nm, 0.663039 km/s) to yield chi^2/dof=1 reduceArc.fitDetectorMap INFO: Softened fit: chi2=1967.833799 dof=1976 xRMS=0.033413 yRMS=0.035226 (0.003024 nm, 1.128881 km/s) from 1016/1477 lines reduceArc.fitDetectorMap INFO: Softened fit quality from reserved lines: chi2=6280.993369 xRMS=0.053251 yRMS=0.049719 (0.004268 nm, 1.593337 km/s) from 148 lines (10.0%) ingestPfsCalibs.py /projects/HSC/PFS/Subaru --output=/tigress/HSC/PFS/Subaru/CALIB-price --validity=1800 --doraise --mode=copy --config clobber=True -- /projects/HSC/PFS/Subaru/rerun/price/pipe2d-709/detectorMap/DETECTORMAP/*.fits </pre> </div></div> Development Rank 0|zzs414: