[PIPE2D-1196] Bootstrap a detectorMap failed Created: 30/Mar/23  Updated: 13/Apr/23  Resolved: 13/Apr/23

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: fmadec Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Reviewers: price

 Description   

about bootstrapping detectorMap:
the recent change of pfsConfig which now include the engineering fibers was not propagated (that's my guess) everywhere.
so the number of fiber really illuminated does not match with pfsConfig

output for n2 at LAM:
I used 10 fibers

bootstrap.profiles WARN: Failed to identify 24 fiberIds: [653, 683, 696, 743, 762, 788, 835, 874, 880, 924, 940, 967, 990, 1033, 1069, 1077, 1122, 1166, 1169, 1211, 1258, 1272, 1301, 1302]
bootstrap.profiles INFO: Profiled 10 fibers
bootstrap INFO: Found 10 fibers on flat

but there is an other issue, because there is 16 engineering fibers so we should have 26 fibers reported by pfsConfig, 2 are missing (652 and 1033)

eng. fiberIds are : ([ 652, 696, 743, 788, 835, 880, 924, 967, 987, 1033, 1077, 1122, 1166, 1211, 1258, 1302] for sm2

 Comments   
Comment by arnaud.lefur [ 31/Mar/23 ]

can you post your command ? so we can reproduce the problem ?

Comment by price [ 31/Mar/23 ]

And the log with backtrace that shows that the command failed.

Comment by fmadec [ 31/Mar/23 ] 
(lsst-scipipe-3.0.0) [fmadec@luigi ~]$ bootstrapDetectorMap.py /data/drp/sps --calib=/data/drp/sps/CALIB/ --rerun fmadec/n2/bootstrap --flatId visit=48749 arm=n --arcId visit=48738 arm=n spectrograph=2 --no-versions --clobber-config -c isr.doBias=False isr.doDark=False isr.doFlat=False isr.doLinearize=False isr.doIPC=True profiles.findThreshold=800 profiles.associationDepth=20 profiles.convolveGrowMask=2 profiles.pruneMinFrac=0.6 findLines.doSubtractContinuum=False 


root INFO: Loading config overrride file '/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/obs_pfs/w.2023.09/config/bootstrap.py' /software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsButler). It will be removed sometime after v23.0 but no earlier than the end of 2021. namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs) /software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsMapper). It will be removed sometime after v23.0 but no earlier than the end of 2021. namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs) lsst.CameraMapper INFO: Loading exposure registry from /data/drp/sps/registry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /data/drp/sps/CALIB/calibRegistry.sqlite3 lsst.CameraMapper INFO: Loading calib registry from /data/drp/sps/CALIB/calibRegistry.sqlite3 root INFO: Running: /software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2023.09/bin/bootstrapDetectorMap.py /data/drp/sps --calib=/data/drp/sps/CALIB/ --rerun fmadec/n2/bootstrap --flatId visit=48749 arm=n --arcId visit=48738 arm=n spectrograph=2 --no-versions --clobber-config -c isr.doBias=False isr.doDark=False isr.doFlat=False isr.doLinearize=False isr.doIPC=True profiles.findThreshold=800 profiles.associationDepth=20 profiles.convolveGrowMask=2 profiles.pruneMinFrac=0.6 findLines.doSubtractContinuum=False bootstrap.isr INFO: Performing ISR on sensor {'visit': 48749, 'arm': 'n', 'dateObs': '2023-03-27', 'site': 'L', 'category': 'B', 'spectrograph': 2, 'field': 'TEST', 'ccd': 5, 'filter': 'n', 'expTime': 10.0, 'dataType': 'FLAT', 'taiObs': '2023-03-27T15:49:32.097', 'pfsDesignId': 2743916541777272600, 'slitOffset': 0.0} bootstrap.profiles WARN: Failed to identify 24 fiberIds: [653, 683, 696, 743, 762, 788, 835, 874, 880, 924, 940, 967, 990, 1033, 1069, 1077, 1122, 1166, 1169, 1211, 1258, 1272, 1301, 1302] bootstrap.profiles INFO: Profiled 10 fibers bootstrap INFO: Found 10 fibers on flat bootstrap FATAL: Failed on dataId={'visit': 48749, 'arm': 'n', 'dateObs': '2023-03-27', 'site': 'L', 'category': 'B', 'spectrograph': 2, 'field': 'TEST', 'ccd': 5, 'filter': 'n', 'expTime': 10.0, 'dataType': 'FLAT', 'taiObs': '2023-03-27T15:49:32.097', 'pfsDesignId': 2743916541777272600, 'slitOffset': 0.0}: RuntimeError: Insufficient traces (10) found vs expected number of fibers (0) Traceback (most recent call last): File "/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/cmdLineTask.py", line 436, in __call__ result = self.runTask(task, dataRef, kwargs) File "/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/cmdLineTask.py", line 504, in runTask return task.runDataRef(dataRef, **kwargs) File "/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2023.09/python/pfs/drp/stella/bootstrap.py", line 143, in runDataRef traces = self.traceFibers(flatRef, flatConfig) File "/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/drp_stella/w.2023.09/python/pfs/drp/stella/bootstrap.py", line 191, in traceFibers raise RuntimeError("Insufficient traces (%d) found vs expected number of fibers (%d)" % RuntimeError: Insufficient traces (10) found vs expected number of fibers (0)
Comment by price [ 31/Mar/23 ]

Why is it expecting to find zero fibers? What's in the pfsConfig?

Comment by fmadec [ 31/Mar/23 ]

it first says:

pfsDesignId': 2743916541777272600 :

bootstrap.profiles WARN: Failed to identify 24 fiberIds: [653, 683, 696, 743, 762, 788, 835, 874, 880, 924, 940, 967, 990, 1033, 1069, 1077, 1122, 1166, 1169, 1211, 1258, 1272, 1301, 1302]

why it then says 0 fibers, I don't know...

 

Comment by arnaud.lefur [ 31/Mar/23 ]

I think it's because of the new targetType DCB

BootstrapConfig define the expected targetTypes to allow

targetType = ListField(dtype=str, default=["SCIENCE", "SKY", "FLUXSTD", "SUNSS_IMAGING", "SUNSS_DIFFUSE"],
                           doc="Target types to allow")

We just need to add "DCB" in there.

Comment by arnaud.lefur [ 01/Apr/23 ]

I build on the branch and the run the command again, this error no longer comes up.
I have another error though : ValueError: On entry to DLASCL parameter number 4 had an illegal value

(lsst-scipipe-3.0.0) [pfs@mario drp_stella]$ bootstrapDetectorMap.py /data/drp/sps --calib=/data/drp/sps/CALIB/ --rerun alefur/n2/bootstrap --flatId visit=48749 arm=n --arcId visit=48738 arm=n spectrograph=2 --no-versions --clobber-config -c isr.doBias=False isr.doDark=False isr.doFlat=False isr.doLinearize=False isr.doIPC=True profiles.findThreshold=800 profiles.associationDepth=20 profiles.convolveGrowMask=2 profiles.pruneMinFrac=0.6 findLines.doSubtractContinuum=False                                                                                                                                      
root INFO: Loading config overrride file '/software/devel/alefur/obs_pfs/config/bootstrap.py'                                                                                                              
/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsButler)
. It will be removed sometime after v23.0 but no earlier than the end of 2021.                                                                                                                             
  namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)
/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/argumentParser.py:778: FutureWarning: Gen2 Butler has been deprecated (PfsMapper)
. It will be removed sometime after v23.0 but no earlier than the end of 2021.
  namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)                                                                                                                                     
lsst.CameraMapper INFO: Loading exposure registry from /data/drp/sps/registry.sqlite3
lsst.CameraMapper INFO: Loading calib registry from /data/drp/sps/CALIB/calibRegistry.sqlite3                                                                                                              
lsst.CameraMapper INFO: Loading calib registry from /data/drp/sps/CALIB/calibRegistry.sqlite3
root INFO: Running: /software/devel/alefur/drp_stella/bin/bootstrapDetectorMap.py /data/drp/sps --calib=/data/drp/sps/CALIB/ --rerun alefur/n2/bootstrap --flatId visit=48749 arm=n --arcId visit=48738 arm=n spectrograph=2 --no-versions --clobber-config -c isr.doBias=False isr.doDark=False isr.doFlat=False isr.doLinearize=False isr.doIPC=True profiles.findThreshold=800 profiles.associationDepth=20 profiles.convolveGrowMask=2 profiles.pruneMinFrac=0.6 findLines.doSubtractContinuum=False                                                                                                                         
bootstrap.isr INFO: Performing ISR on sensor {'visit': 48749, 'arm': 'n', 'dateObs': '2023-03-27', 'site': 'L', 'category': 'B', 'spectrograph': 2, 'field': 'TEST', 'ccd': 5, 'filter': 'n', 'expTime': 10.0, 'dataType': 'FLAT', 'taiObs': '2023-03-27T15:49:32.097', 'pfsDesignId': 2743916541777272600, 'slitOffset': 0.0}                                                 
bootstrap.profiles WARN: Failed to identify 24 fiberIds: [653, 683, 696, 743, 762, 788, 835, 874, 880, 924, 940, 967, 990, 1033, 1069, 1077, 1122, 1166, 1169, 1211, 1258, 1272, 1301, 1302]
bootstrap.profiles INFO: Profiled 10 fibers                                                          
bootstrap INFO: Found 10 fibers on flat                                                                                                                                                                    
bootstrap.isr INFO: Performing ISR on sensor {'visit': 48738, 'arm': 'n', 'spectrograph': 2, 'dateObs': '2023-03-27', 'site': 'L', 'category': 'B', 'field': 'TEST', 'ccd': 5, 'filter': 'n', 'expTime': 18.0, 'dataType': 'COMPARISON', 'taiObs': '2023-03-27T15:10:45.436', 'pfsDesignId': 2743916541777272600, 'slitOffset': 0.0}
bootstrap.readLineList INFO: Filtered line lists, keeping species {'KrI'}.                                                                                                                                 
bootstrap INFO: Found 2341 lines in 10 traces
bootstrap INFO: Matched 629 lines
bootstrap INFO: Median difference from detectorMap: nan,nan pixels
bootstrap FATAL: Failed on dataId={'visit': 48749, 'arm': 'n', 'dateObs': '2023-03-27', 'site': 'L', 'category': 'B', 'spectrograph': 2, 'field': 'TEST', 'ccd': 5, 'filter': 'n', 'expTime': 10.0, 'dataType': 'FLAT', 'taiObs': '2023-03-27T15:49:32.097', 'pfsDesignId': 2743916541777272600, 'slitOffset': 0.0}: ValueError: On entry to DLASCL parameter number 4 had an illegal value
Traceback (most recent call last):
  File "/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/cmdLineTask.py", line 436, in __call__
    result = self.runTask(task, dataRef, kwargs)
  File "/software/stack/20220609/stack/miniconda3-py38_4.9.2-3.0.0/Linux64/pipe_base/g590c34a36e+5da9528084/python/lsst/pipe/base/cmdLineTask.py", line 504, in runTask
    return task.runDataRef(dataRef, **kwargs)
  File "/software/devel/alefur/drp_stella/python/pfs/drp/stella/bootstrap.py", line 153, in runDataRef
    self.fitDetectorMap(matches, lineResults.detectorMap, fiberId)
  File "/software/devel/alefur/drp_stella/python/pfs/drp/stella/bootstrap.py", line 425, in fitDetectorMap
    result = fitChebyshev2D(xx, yy, self.config.spatialOrder, self.config.spectralOrder,
  File "/software/devel/alefur/drp_stella/python/pfs/drp/stella/bootstrap.py", line 609, in fitChebyshev2D
    xFit = fitter(model, xx[0][good], xx[1][good], yy[0][good])
  File "/software/stack/20220609/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-3.0.0/lib/python3.8/site-packages/astropy/modeling/fitting.py", line 269, in wrapper
    return func(self, model, x, y, z=z, **kwargs)
  File "/software/stack/20220609/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-3.0.0/lib/python3.8/site-packages/astropy/modeling/fitting.py", line 771, in __call__
    lacoef, resids, rank, sval = np.linalg.lstsq(lhs[good],
  File "<__array_function__ internals>", line 5, in lstsq
  File "/software/stack/20220609/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-3.0.0/lib/python3.8/site-packages/numpy/linalg/linalg.py", line 2305, in lstsq
    x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj)
ValueError: On entry to DLASCL parameter number 4 had an illegal value
Comment by price [ 04/Apr/23 ]

NANs were getting into the positions used for the fit, presumably from the detectorMap being given wavelengths slightly off the detector. I've added a change to exclude these.

Comment by arnaud.lefur [ 13/Apr/23 ]

merged to master

Generated at Sat Feb 10 16:04:06 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.