[PIPE2D-633] Debug false lines in extracted spectra Created: 29/Sep/20 Updated: 29/Sep/20 Resolved: 29/Sep/20 |
|
| Status: | Done |
| Project: | DRP 2-D Pipeline |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Normal |
| Reporter: | price | Assignee: | price |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Description |
|
|
| Comments |
| Comment by rhl [ 29/Sep/20 ] |
|
So you're looking at why the 2-D data isn't significant, but the 1-D result is? Is this related to the problem that we use the background as an estimate for its variance? (population v. sample)? |
| Comment by price [ 29/Sep/20 ] |
|
That's the first thing I'm investigating. There may be other causes that I will chase down as well. I don't think this is related to how the variance is measured. At the moment, the question is how a single-pixel value of -0.14 with a variance of 21 gets extracted to a spectrum value of -1.0 with a variance of 0.11. |
| Comment by price [ 29/Sep/20 ] |
|
Ah, I had mismatched the image and spectrum. The actual image value is -10.8 and variance of 12. That's a S/N of about 3, which matches the S/N in the spectrum. So that question is answered. Now, how is that single 3-sigma low value leaking into everything else? |
| Comment by price [ 29/Sep/20 ] |
|
The low value is in a sky fiber, and so the low value is an ingredient for the 1D sky subtraction model, and the subtraction of a low value makes a high value. The 1D sky subtraction is using FitFocalPlaneTask.fit, the docstring for which says:
This implementation is a placeholder, as it simply averages the input
vectors instead of doing any real fitting or rejection of outliers, and
no attention is paid to the position of the fibers on the focal plane.
I should add that there's no treatment of errors either. So I think I now understand what's causing the problem, and the solution is to replace FitFocalPlaneTask with a proper implementation. We'll always be limited by the mere two sky fibers in the integration test, but we should be able to make that single bad pixel go away. |
| Comment by rhl [ 29/Sep/20 ] |
|
If we need to add more sky fibres, let's do it. I'd be OK with a 50:50 ratio for a test suite (as opposed to a guessed 20:80 sky:science on the sky) |
| Comment by price [ 29/Sep/20 ] |
|
Debugging is done. Will be fixed on |