[PIPE2D-507] ConstructFiberFlat.py hangs against Subaru data Created: 17/Jan/20 Updated: 13/Feb/20 Resolved: 12/Feb/20 |
|
| Status: | Done |
| Project: | DRP 2-D Pipeline |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Major |
| Reporter: | hassan | Assignee: | hassan |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Story Points: | 5 | ||||||||||||||||
| Sprint: | 2DDRP-2020 A | ||||||||||||||||
| Reviewers: | price | ||||||||||||||||
| Description |
|
As reported by cloomis in the #commissioning slack channel, the fiber flat construction using recent SM1 data taken at Subaru hangs indefinitely. An example command invocation is as follows: constructFiberFlat.py $ROOT --calib $ROOT/CALIB --rerun calib0 --id visit=123..251:3 --batch-type=smp --cores 2 --doraise |
| Comments |
| Comment by hassan [ 17/Jan/20 ] |
|
The cause of the hanging is the presence of NaN values in the postISRCCD image and variance arrays, that are subsequently passed to the C++ function findCenterPositionsOneTrace and to the MPFitGaussLim fitting routine, which hangs. Replacing the NaN values in the images and variance arrays with non-NaN values seems to fix the problem, but further investigation is required. The cause of the NaN values appearing in the postISRCCD image data also needs to be investigated (most likely as a separate ticket). As a fix is needed in order for the LAM team to process Subaru data effectively, a workaround is being developed. |
| Comment by hassan [ 17/Jan/20 ] |
|
In should be noted that the image mask flags pixels with NaN values correctly, however the findCenterPositionsOneTrace code does not use those mask arrays currently. Also note that after a temporary fix to the NaN values describe in the earlier comment, the presence of (close to) zero values in the variance array causes a subsequent exception in the C++ CurveFitting::fitProfile1d function. This is also being investigated and a fix is expected soon. |
| Comment by rhl [ 17/Jan/20 ] |
|
Did you run the standard replace-NaNs code, or do something else? When you look at the images, does something jump out at you about which pixels have NaNs? The images can't |
| Comment by price [ 17/Jan/20 ] |
|
I believe this is fixed on |
| Comment by hassan [ 29/Jan/20 ] |
|
Fix implemented based on changes to drp_stella originally put forward by P Price in |
| Comment by price [ 30/Jan/20 ] |
|
Minor issues reported on the GitHub PR. |
| Comment by hassan [ 12/Feb/20 ] |
|
Fix merged to master. |