[PIPE2D-841] constructPfsDark can silently generate an all-NaN master dark

Thu, 17 Jun 2021 15:31:16 +0000

I was looking at a curious defect that's present in e.g. 11150 but not in the darks created using 12137.. 12166, so ran the command

constructPfsDark.py /projects/HSC/PFS/LAM --rerun rhl/dark --id visit=11510..11514 -j 10 --batch-type none  -c isr.doBias=False

(there are no biases registered for that data, but that's a different problem). When I went to look at the resulting master bias it turned out to be all NaN with NO_DATA set.

Looking back at the logs there are signs of trouble:

dark INFO: Scatter combination
dark INFO: Combining {'arm': 'r', 'spectrograph': 1, 'ccd': 1, 'filter': 'NONE', 'calibTime': '2019-02-01T19:56:44.999998720', 'visit0': 11510, 'calibDate': '2019-02-01'} on tiger2-sumire.princeton.edu:212211
/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
/tigress/HSC/PFS/stack/20190925/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/site-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in true_divide
  ret = ret.dtype.type(ret / rcount)
dark INFO: Writing {'arm': 'r', 'spectrograph': 1, 'ccd': 1, 'filter': 'NONE', 'calibTime': '2019-02-01T19:56:44.999998720', 'visit0': 11510, 'calibDate': '2019-02-01'} on tiger2-sumire.princeton.edu:212211

All the visits look fine by eye but even if I did something stupid we need to improve the diagnostics. If this is all going to be replaced with new code from LSST that'd be OK too, but the fact that we have PFS-flavoured versions of these routines makes me a little concerned that that won't be a panacea.

PFS-JIRA

[PIPE2D-841] constructPfsDark can silently generate an all-NaN master dark