[PIPE2D-507] ConstructFiberFlat.py hangs against Subaru data Created: 17/Jan/20  Updated: 13/Feb/20  Resolved: 12/Feb/20

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Major
Reporter: hassan Assignee: hassan
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by PIPE2D-481 Problems generating flat calibration ... Won't Fix
Relates
relates to PIPE2D-512 ReduceArc.py fails on Subaru data usi... Done
Story Points: 5
Sprint: 2DDRP-2020 A
Reviewers: price

 Description   

As reported by cloomis in the #commissioning slack channel, the fiber flat construction using recent SM1 data taken at Subaru hangs indefinitely.

An example command invocation is as follows:

constructFiberFlat.py $ROOT --calib $ROOT/CALIB --rerun calib0 --id visit=123..251:3 --batch-type=smp --cores 2 --doraise


 Comments   
Comment by hassan [ 17/Jan/20 ]

The cause of the hanging is the presence of NaN values in the postISRCCD image and variance arrays, that are subsequently passed to the C++ function

findCenterPositionsOneTrace

and to the

MPFitGaussLim

fitting routine, which hangs.

Replacing the NaN values in the images and variance arrays with non-NaN values seems to fix the problem, but further investigation is required.

The cause of the NaN values appearing in the postISRCCD image data also needs to be investigated (most likely as a separate ticket).

As a fix is needed in order for the LAM team to process Subaru data effectively, a workaround is being developed.

Comment by hassan [ 17/Jan/20 ]

In should be noted that the image mask flags pixels with NaN values correctly, however the findCenterPositionsOneTrace code does not use those mask arrays currently.

Also note that after a temporary fix to the NaN values describe in the earlier comment, the presence of (close to) zero values in the variance array causes a subsequent exception in the C++ CurveFitting::fitProfile1d function. This is also being investigated and a fix is expected soon.

Comment by rhl [ 17/Jan/20 ]

Did you run the standard replace-NaNs code, or do something else? When you look at the images, does something jump out at you about which pixels have NaNs? The images can't have NaNs unless there is some problem in the calibration frame; you should check that too

Comment by price [ 17/Jan/20 ]

I believe this is fixed on PIPE2D-391, which is ready for merge but blocked by conflicts that I need to sort out. I suggest that in the mean time, we cherry-pick the fix from obs_pfs and drp_stella.

Comment by hassan [ 29/Jan/20 ]

Fix implemented based on changes to drp_stella originally put forward by P Price in PIPE2D-391, plus variance plane update.
Added additional fix to low variance values, that are not masked. These are set according to a config parameter minVarRN, which defines the minimum variance in units of the appropriate amplifier read-out noise.

Comment by price [ 30/Jan/20 ]

Minor issues reported on the GitHub PR.

Comment by hassan [ 12/Feb/20 ]

Fix merged to master.
Separate related problems (eg PIPE2D-512) will be addressed in separate tickets.

Generated at Sat Feb 10 15:54:12 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.