[PIPE2D-864] bootstrap does not identify lines correctly for med-res data Created: 28/Jun/21 Updated: 10/Jul/21 Resolved: 10/Jul/21 |
|
| Status: | Done |
| Project: | DRP 2-D Pipeline |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Normal |
| Reporter: | hassan | Assignee: | price |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
bootstrap medRes.png
bootstrap-red-DCB.png
|
||||||||
| Issue Links: |
|
||||||||
| Story Points: | 2 | ||||||||
| Sprint: | 2DDRP-2021 A 6 | ||||||||
| Reviewers: | hassan | ||||||||
| Description |
|
As briefly mentioned during the last DRP telecon 2021-06-18, I am having difficulty generating a bootstrapped detectormap for medium-resolution data. running bootstrapDetectorMap with arc visit=63106 (room_lights, no_screen) and flat visit=63050 with debug on: bootstrapDetectorMap.py /projects/HSC/PFS/Subaru/ --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS-2021-06-16 --rerun hassan/bstrapMed --flatId visit=63050 arm=m --arcId visit=63106 arm=m --debug We see the line identification shown in figure 'bootstrap medRes.png'. The arc lines are clearly apparent, but the crosses do not match any of the lines. This is in contrast to running the same task against DCB r data: bootstrapDetectorMap.py /projects/HSC/PFS/Subaru/ --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS-2021-06-16/ --rerun hassan/bstrapRed --flatId visit=30044 arm=r --arcId visit=30045 arm=r --debug Where the lines are clearly identified. See figure 'bootstrap-red-DCB.png' . Also no problems with identification noticed when using simulated arcs from the integration test. |
| Comments |
| Comment by price [ 29/Jun/21 ] |
|
There are astropy fit warnings and abysmal RMS values coming from bootstrapDetectorMap.py when running on m SuNSS data. $ bootstrapDetectorMap.py /projects/HSC/PFS/Subaru/ --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS-2021-06-16 --rerun price/pipe2d-864 --flatId visit=63050 arm=m --arcId visit=63106 arm=m [...] WARNING: The fit may be unsuccessful; check fit_info['message'] for more information. [astropy.modeling.fitting] astropy WARN: The fit may be unsuccessful; check fit_info['message'] for more information. WARNING: The fit may be unsuccessful; check fit_info['message'] for more information. [astropy.modeling.fitting] astropy WARN: The fit may be unsuccessful; check fit_info['message'] for more information. bootstrap INFO: Found 6586 lines in 254 traces bootstrap INFO: Matched 5617 lines bootstrap INFO: Median difference from detectorMap: -5.012633,0.618370 pixels bootstrap INFO: Fit 1340/2859 points, rms: x=4.320284 y=6.319983 total=3.232250 pixels bootstrap INFO: Updating detectorMap... bootstrap INFO: Median difference from detectorMap: 5.852661,0.959997 pixels bootstrap INFO: Fit 888/2758 points, rms: x=3.860911 y=4.141929 total=1.879898 pixels bootstrap INFO: Updating detectorMap... |
| Comment by hassan [ 29/Jun/21 ] |
|
Yes, I think the fit is failing due to the poor identification of lines prior to that. |
| Comment by price [ 29/Jun/21 ] |
|
One problem is that no lamps are indicated as being on in the header, and so the software assumes it's looking at the sky. Looking at "dome arcs" is a special case, so we need to configure it specially: config.readLineList.lampList = ["HgI", "KrI", "ArI"] |
| Comment by hassan [ 29/Jun/21 ] |
|
I had ran config.readLineList.lampList = ["Ar"]' a few days ago with the same problem. Will check that again. |
| Comment by cloomis [ 29/Jun/21 ] |
|
For future acquisitions, ICS could either adjust some of the lamp cards, or add some other PFS-specific cards. Note that DATA-TYP cannot be sensible. The only available choices are: ACQUISITION, BIAS, COMPARISON, DARK, DOMEFLAT, DOMEFLAT_ON, DOMEFLAT_OFF, FLAT, FOCUSING, INSTLAT, OBJECT, SKYFLAT, STANDARD, STANDARD_STAR, TEST Actually, we are planning to add a room light card, based on the parasite data we are getting from the telescope while running SuNSS; I'll let you know what that is when we get there. |
| Comment by price [ 01/Jul/21 ] |
|
The main problems were identifying the lamps being used (the code defaults to assuming sky if no lamps are set in the FITS header) and identifying a good set of fibers. With the following configuration overrides: config.readLineList.lampList = ["HgI", "KrI", "ArI"] # otherwise we get sky config.spectralOffset = -20 # by-eye estimate config.profiles.profileRadius = 3 # because we have some close-packed fibers # The following give us only the good fibers, which give us a reliable fiber identification from pfs.datamodel import FiberStatus config.fiberStatus = [FiberStatus.GOOD] config.profiles.findThreshold = 2000 # convolution is probably picking up a lot of scattered light I get a pretty good bootstrap result:
bootstrap.readLineList INFO: Filtered line lists for lamps: ArI,HgI,KrI
bootstrap INFO: Found 6748 lines in 251 traces
bootstrap INFO: Matched 6741 lines
bootstrap INFO: Median difference from detectorMap: 6.389995,4.018591 pixels
bootstrap INFO: Fit 2983/3309 points, rms: x=0.056048 y=0.173571 total=0.126654 pixels
bootstrap INFO: Updating detectorMap...
bootstrap INFO: Median difference from detectorMap: 5.759607,5.856603 pixels
bootstrap INFO: Fit 2969/3432 points, rms: x=0.067750 y=0.108061 total=0.075888 pixels
bootstrap INFO: Updating detectorMap...
|
| Comment by hassan [ 10/Jul/21 ] |
|
I am able to reproduce Paul's findings above, and from eye-balling the lines predicted by the resultant detectormap, the results are much improved (sub-pixel error in both x and y). Minor comment added pull request. |
| Comment by hassan [ 10/Jul/21 ] |
|
Command line executed: bootstrapDetectorMap.py /projects/HSC/PFS/Subaru/ --calib /projects/HSC/PFS/Subaru/CALIB-SuNSS-2021-06-16 --rerun hassan/bstrap-pipe2d-864 --flatId visit=63050 arm=m --arcId visit=63106 arm=m --debug --configfile pipe2d-864.config with the file pipe2d-864.config containing the contents Paul has already listed above. |
| Comment by price [ 10/Jul/21 ] |
|
Merged. |