[PIPE2D-1346] Generate detectorMaps for Dec. 2023 data Created: 26/Jan/24  Updated: 08/Feb/24

Status: In Progress
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: Kiyoto Yabe Assignee: Kiyoto Yabe
Resolution: Unresolved Votes: 0
Labels: EngRun
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2024-02-07 at 10.21.57.png     PNG File Screenshot 2024-02-07 at 10.22.04.png    
Sprint: PreEngRun15Mar

 Description   

Generate detectorMaps based on the calibration data taken during the telescope test in Dec. 2023.



 Comments   
Comment by Kiyoto Yabe [ 02/Feb/24 ]

After m4 bootstrap detectorMap, I just ran reduceArc.py for all camera using calib set on Dec. 28 with and without doSlitOffsets option. 

Without doSlitOffsets:

 (b1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=45654117.632993 dof=11091684 xRMS=0.026695 yRMS=0.083563 xSoften=0.024577 ySoften=0.098187 from 11033952 lines (KrI: 6792, NeI: 49790, Trace: 10976142, XeI: 1228)
 (b2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=43612508.240033 dof=10598788 xRMS=0.028245 yRMS=0.040853 xSoften=0.026353 ySoften=0.049451 from 10542814 lines (KrI: 7325, NeI: 46933, Trace: 10486762, XeI: 1794)
 (b3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=29671618.184812 dof=10929293 xRMS=0.024581 yRMS=0.040523 xSoften=0.020406 ySoften=0.043460 from 10872801 lines (KrI: 7536, NeI: 47154, Trace: 10816231, XeI: 1880)
 (b4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=48864539.510907 dof=11140335 xRMS=0.030246 yRMS=0.047646 xSoften=0.027864 ySoften=0.050703 from 11080571 lines (KrI: 8639, NeI: 48748, Trace: 11020729, XeI: 2455)
 (r1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=72012855.325722 dof=14317177 xRMS=0.022759 yRMS=0.025278 xSoften=0.023066 ySoften=0.025133 from 13996892 lines (ArI: 76722, KrI: 32487, NeI: 135019, Trace: 13676529, XeI: 76135)
 (r2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=53442423.869833 dof=13336196 xRMS=0.022727 yRMS=0.031812 xSoften=0.021971 ySoften=0.031884 from 13041169 lines (ArI: 74087, KrI: 30759, NeI: 123990, Trace: 12746064, XeI: 66269)
 (r3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=67323468.655685 dof=13619054 xRMS=0.025899 yRMS=0.028046 xSoften=0.025126 ySoften=0.027540 from 13324369 lines (ArI: 76542, KrI: 31910, NeI: 127131, Trace: 13029606, XeI: 59180)
 (r4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=113267922.604893 dof=13907089 xRMS=0.031139 yRMS=0.031577 xSoften=0.031652 ySoften=0.030808 from 13592573 lines (ArI: 77098, KrI: 33121, NeI: 133876, Trace: 13277979, XeI: 70499)
 (n1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=32358843.968862 dof=10738544 xRMS=0.026294 yRMS=0.051225 xSoften=0.023504 ySoften=0.045070 from 10626768 lines (ArI: 53078, KrI: 9598, NeI: 27738, Trace: 10514920, XeI: 21434)
 (n2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=163776230.150713 dof=10195383 xRMS=0.042713 yRMS=0.063358 xSoften=0.046465 ySoften=0.056897 from 10075067 lines (ArI: 55640, KrI: 10470, NeI: 27821, Trace: 9954679, XeI: 26457)
 (n3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=115193395.159363 dof=11399862 xRMS=0.033644 yRMS=0.064554 xSoften=0.034658 ySoften=0.059119 from 11269761 lines (ArI: 60106, KrI: 10440, NeI: 31325, Trace: 11139588, XeI: 28302)
 (m1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=173954239.899078 dof=13539282 xRMS=0.049939 yRMS=0.060598 xSoften=0.048119 ySoften=0.062298 from 13333456 lines (ArI: 45788, KrI: 37450, NeI: 77198, Trace: 13127552, XeI: 45468)
 (m2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=58837302.710912 dof=12462895 xRMS=0.029090 yRMS=0.106042 xSoften=0.024967 ySoften=0.107446 from 12285333 lines (ArI: 41088, KrI: 33998, NeI: 68965, Trace: 12107693, XeI: 33589)
 (m3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=150178349.046475 dof=13126397 xRMS=0.046668 yRMS=0.172309 xSoften=0.042884 ySoften=0.175301 from 12933802 lines (ArI: 43618, KrI: 35265, NeI: 74917, Trace: 12741129, XeI: 38873)
 (m4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=225006469.381380 dof=13544467 xRMS=0.060372 yRMS=0.047453 xSoften=0.059073 ySoften=0.047816 from 13339174 lines (ArI: 46204, KrI: 36962, NeI: 77463, Trace: 13133803, XeI: 44742)

With doSlitOffsets:

 (b1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=31463086.579820 dof=10682450 xRMS=0.022398 yRMS=0.140902 xSoften=0.019733 ySoften=0.193647 from 10653234 lines (KrI: 4064, NeI: 25898, Trace: 10622750, XeI: 522)
 (b2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=26702337.438333 dof=10206638 xRMS=0.023049 yRMS=0.027032 xSoften=0.021102 ySoften=0.038639 from 10163701 lines (KrI: 6960, NeI: 35404, Trace: 10119524, XeI: 1813)
 (b3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=22509320.593434 dof=10763935 xRMS=0.021811 yRMS=0.028973 xSoften=0.017355 ySoften=0.031570 from 10718023 lines (KrI: 6517, NeI: 38915, Trace: 10670833, XeI: 1758)
 (b4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=24630535.023649 dof=10919160 xRMS=0.021947 yRMS=0.029005 xSoften=0.018053 ySoften=0.037566 from 10871340 lines (KrI: 7715, NeI: 38968, Trace: 10822248, XeI: 2409)
 (r1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=35946055.807622 dof=13511162 xRMS=0.016787 yRMS=0.018428 xSoften=0.015874 ySoften=0.016581 from 13224003 lines (ArI: 66477, KrI: 26740, NeI: 121777, Trace: 12935576, XeI: 73433)
 (r2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=32071219.337058 dof=12800928 xRMS=0.018357 yRMS=0.019742 xSoften=0.016766 ySoften=0.017297 from 12535687 lines (ArI: 64411, KrI: 26406, NeI: 111256, Trace: 12269206, XeI: 64408)
 (r3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=31030301.416250 dof=13023263 xRMS=0.018259 yRMS=0.018566 xSoften=0.016251 ySoften=0.014434 from 12758697 lines (ArI: 68154, KrI: 27658, NeI: 113596, Trace: 12492853, XeI: 56436)
 (r4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=29934782.534349 dof=13184414 xRMS=0.016616 yRMS=0.018485 xSoften=0.014679 ySoften=0.014756 from 12907391 lines (ArI: 63809, KrI: 25777, NeI: 121589, Trace: 12629096, XeI: 67120)
 (n1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=24881715.186988 dof=10531070 xRMS=0.023337 yRMS=0.041044 xSoften=0.019363 ySoften=0.033661 from 10424378 lines (ArI: 51573, KrI: 9336, NeI: 27182, Trace: 10316424, XeI: 19863)
 (n2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=30555501.108077 dof=7792104 xRMS=0.021727 yRMS=0.044413 xSoften=0.023384 ySoften=0.041866 from 7687007 lines (ArI: 48512, KrI: 9057, NeI: 26076, Trace: 7580676, XeI: 22686)
 (n3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=34396254.833888 dof=9368745 xRMS=0.020619 yRMS=0.047238 xSoften=0.020070 ySoften=0.051553 from 9254990 lines (ArI: 52328, KrI: 9207, NeI: 29943, Trace: 9139963, XeI: 23549)
 (m1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=29341520.983447 dof=12832557 xRMS=0.022041 yRMS=0.018298 xSoften=0.018546 ySoften=0.017795 from 12659409 lines (ArI: 31228, KrI: 30013, NeI: 70219, Trace: 12484993, XeI: 42956)
 (m2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=25225599.834047 dof=12028787 xRMS=0.023027 yRMS=0.019033 xSoften=0.018720 ySoften=0.017868 from 11877184 lines (ArI: 29968, KrI: 28561, NeI: 63768, Trace: 11724341, XeI: 30546)
 (m3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=27701383.752870 dof=12709725 xRMS=0.024453 yRMS=0.023058 xSoften=0.019908 ySoften=0.021540 from 12552180 lines (ArI: 23842, KrI: 28960, NeI: 69417, Trace: 12393357, XeI: 36604)
 (m4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=26377082.509995 dof=12807863 xRMS=0.021994 yRMS=0.020176 xSoften=0.017484 ySoften=0.018413 from 12644289 lines (ArI: 27379, KrI: 26404, NeI: 70672, Trace: 12479443, XeI: 40391)

The results are basically good for doSlitOffsets=True except for b1.
The script is in /work/kiyoyabe/drp/calibs/generateCALIBs/2024jan/update_detectorMaps/process.sh and the reruns are kiyoyabe/calibs/2024jan/20240201 and kiyoyabe/calibs/2024jan/20240201-2.

Comment by Kiyoto Yabe [ 03/Feb/24 ]

detectorMaps before 12/28 are also regenerated.

With doSlitOffsets:

(b1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=32523615.517450 dof=10762267 xRMS=0.022305 yRMS=0.153569 xSoften=0.019694 ySoften=0.190640 from 10728639 lines (KrI: 8278, NeI: 26274, Trace: 10693739, XeI: 348)
(b2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=27030738.188432 dof=10398006 xRMS=0.023096 yRMS=0.029890 xSoften=0.021252 ySoften=0.042375 from 10347914 lines (KrI: 13875, NeI: 36229, Trace: 10296550, XeI: 1260)
(b3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=22748128.768425 dof=10813851 xRMS=0.021693 yRMS=0.030402 xSoften=0.017388 ySoften=0.031811 from 10755149 lines (ArI: 8, KrI: 17264, NeI: 40542, Trace: 10695167, XeI: 2168)
(b4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=22402765.524722 dof=10881889 xRMS=0.023161 yRMS=0.034262 xSoften=0.017946 ySoften=0.043064 from 10831477 lines (KrI: 14232, NeI: 35754, Trace: 10779787, XeI: 1704)
(r1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=37396790.988725 dof=14062377 xRMS=0.016862 yRMS=0.017485 xSoften=0.016119 ySoften=0.016375 from 13786992 lines (ArI: 46077, KrI: 47332, NeI: 116812, Trace: 13510333, XeI: 66438)
(r2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=34545617.146477 dof=13515894 xRMS=0.018637 yRMS=0.021583 xSoften=0.017415 ySoften=0.020777 from 13237993 lines (ArI: 54226, KrI: 54252, NeI: 111674, Trace: 12958816, XeI: 59025)
(r3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=28147413.885080 dof=13198939 xRMS=0.018625 yRMS=0.017084 xSoften=0.016367 ySoften=0.013627 from 12913846 lines (ArI: 61198, KrI: 55648, NeI: 114642, Trace: 12627473, XeI: 54885)
(r4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=28141029.559018 dof=12605574 xRMS=0.021522 yRMS=0.018036 xSoften=0.020884 ySoften=0.013445 from 12362943 lines (ArI: 37717, KrI: 46454, NeI: 106703, Trace: 12119032, XeI: 53037)
(n1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=25573026.116767 dof=10664263 xRMS=0.023767 yRMS=0.039903 xSoften=0.019844 ySoften=0.033097 from 10533114 lines (ArI: 68309, KrI: 19044, NeI: 27449, Trace: 10400697, XeI: 17615)
(n2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=32718985.073962 dof=9062994 xRMS=0.022129 yRMS=0.042836 xSoften=0.024605 ySoften=0.039836 from 8940265 lines (ArI: 58912, KrI: 18041, NeI: 26083, Trace: 8816268, XeI: 20961)
(n3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=35207348.435563 dof=9548186 xRMS=0.020590 yRMS=0.054249 xSoften=0.020164 ySoften=0.058282 from 9402655 lines (ArI: 69100, KrI: 21401, NeI: 31453, Trace: 9255848, XeI: 24853)
(m1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=35981124.599212 dof=12425009 xRMS=0.016091 yRMS=0.038190 xSoften=0.014710 ySoften=0.044385 from 12268153 lines (ArI: 14738, KrI: 28032, NeI: 70311, Trace: 12110025, XeI: 45047)
(m2)(fitDistortedDetectorMap.py:1337)- Final result: chi2=32239518.853053 dof=12092425 xRMS=0.017638 yRMS=0.020846 xSoften=0.016282 ySoften=0.019417 from 11942480 lines (ArI: 16046, KrI: 29789, NeI: 67180, Trace: 11791263, XeI: 38202)
(m3)(fitDistortedDetectorMap.py:1337)- Final result: chi2=34444851.020252 dof=12086124 xRMS=0.019486 yRMS=0.036357 xSoften=0.017721 ySoften=0.037605 from 11946455 lines (ArI: 15260, KrI: 19269, NeI: 70062, Trace: 11805508, XeI: 36356)
(m4)(fitDistortedDetectorMap.py:1337)- Final result: chi2=25689398.117607 dof=12171776 xRMS=0.018939 yRMS=0.039549 xSoften=0.015524 ySoften=0.040543 from 12039506 lines (ArI: 10020, KrI: 23161, NeI: 67097, Trace: 11905958, XeI: 33270)

b1 detectorMap is not good so turned off doSlitOffsets:

 (b1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=43351474.033928 dof=11171213 xRMS=0.025533 yRMS=0.089772 xSoften=0.023151 ySoften=0.104356 from 11107816 lines (KrI: 12677, NeI: 50117, Trace: 11044341, XeI: 681)
Comment by Kiyoto Yabe [ 03/Feb/24 ]

 
detectorMaps after 12/28/23:

camera xRMS (pix) yRMS (pix) doSlitOffsets
b1 0.026695 0.083563 False
b2 0.023049 0.027032 True
b3 0.021811 0.028973 True
b4 0.021947 0.029005 True
r1 0.016787 0.018428 True
r2 0.018357 0.019742 True
r3 0.018259 0.018566 True
r4 0.016616 0.018485 True
n1 0.023337 0.041044 True
n2 0.021727 0.044413 True
n3 0.020619 0.047238 True
m1 0.022041 0.018298 True
m2 0.023027 0.019033 True
m3 0.024453 0.023058 True
m4 0.021994 0.020176 True

 
detectorMaps before 12/28/23:

 

camera xRMS (pix) yRMS (pix) doSlitOffsets
b1 0.025533 0.089772 False
b2 0.023096 0.029890 True
b3 0.021693 0.030402 True
b4 0.023161 0.034262 True
r1 0.016862 0.017485 True
r2 0.018637 0.021583 True
r3 0.018625 0.017084 True
r4 0.021522 0.018036 True
n1 0.023767 0.039903 True
n2 0.022129 0.042836 True
n3 0.020590 0.054249 True
m1 0.016091 0.038190 True
m2 0.017638 0.020846 True
m3 0.019486 0.036357 True
m4 0.018939 0.039549 True

 

 

Comment by Wilfred Gee [ 08/Feb/24 ]

The visit that the DM QA identifies is 104151 in `b1` only.  The Argon visit set is 104149..104153 and all the DMs in that set look the same except for the `b1`.  Attached is visit 104150, which is representative of all the others, and 104151, which is the outlier.

 

Comment by Kiyoto Yabe [ 08/Feb/24 ]

Thank you. Excluding that visit, the result doesn't change much.

(b1)(fitDistortedDetectorMap.py:1337)- Final result: chi2=45636858.630669 dof=11091617 xRMS=0.026689 yRMS=0.083871 xSoften=0.024572 ySoften=0.098460 from 11033843 lines (KrI: 6815, NeI: 49758, Trace: 10975991, XeI: 1279)

 

Generated at Sat Feb 10 16:06:23 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.