[PIPE1D-33] Failed to compute continuum fit for some spectra Created: 09/Nov/20 Updated: 12/Feb/21 Resolved: 12/Feb/21 |
|
| Status: | Done |
| Project: | DRP 1D pipeline |
| Component/s: | drp_1d |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Normal |
| Reporter: | Kiyoto Yabe | Assignee: | Ali Allaoui |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
Running the latest master branch (as of Nov. 9), the following error occurs: (lsst-scipipe-1172c30) [yabe@fe B0]$ cat process_spectra.log 2020-11-09 13:21:23,830 :: INFO :: Running process_spectra 0.14.0 2020-11-09 13:21:23,892 :: INFO :: Loading /gpfs02/work/yabe/pfs/drp/pipe_e2e/regular_processing/pipe1d_results/calibration/linecatalogs/linecatalogamazedvacuum_F1_noHepsilon.txt 2020-11-09 13:21:23,931 :: INFO :: Loading /gpfs02/work/yabe/pfs/drp/pipe_e2e/regular_processing/pipe1d_results/calibration/linecatalogs/linecatalogamazedvacuum_F2_noHepsilon-linemeas.txt 2020-11-09 13:21:23,931 :: INFO :: Loading /gpfs02/work/yabe/pfs/drp/pipe_e2e/regular_processing/pipe1d_results/calibration/templates/BC03_sdss_tremonti21 2020-11-09 13:22:00,255 :: ERROR :: Can't process : Operator-Linemodel: Failed to compute continuum fit. Negative amplitude detected! aborting... (lsst-scipipe-1172c30) [yabe@fe B0]$ cat amazed.log Info: Loaded 21 templates for category galaxy. Info: Processing context initialization Info: Continuum estimation on input spectrum: using IrregularSamplingMedian Info: Continuum estimation - IrregularSamplingMedian - medianKernelWidth = 400.00 Info: =============================================== Info: <proc-spc><pfsObject-00000-00000-0,0-0000000000000004-004-0x44b41c65176b07dc.fits> Info: Processing spc:pfsObject-00000-00000-0,0-0000000000000004-004-0x44b41c65176b07dc.fits (CLambdaRange: 3853.599854-11999.200195:0.800049) Info: Stellar solve enabled : yes Info: Processflow - Loading star templates catalog : templates/stars_templates_vlba37_2018 Info: Loaded 37 templates for category star. Info: stellar-solve: Loaded (category=star) template count = 37 Info: Stellar fitting redshift range = [-0.00100, 0.00100], step=0.000050 Info: Processing stellar fitting Info: Method parameters: Info: -overlapThreshold: 1.000 Info: -component: nocontinuum Info: -interp: precomputedfinegrid Info: -IGM extinction: no Info: -ISM dust-fit: yes Info: -pdfcombination: bestchi2 Info: -saveintermediateresults: 0 Info: Info: Iterating over 1 tplCategories Info: trying star (37 templates) Info: chisquare2solve: Pdfz computation Info: chisquare2solve: using cstLog = 332733.329288 Info: chisquare2solve: Pdfz combination - BestChi2 Info: Found 1 z-candidates Info: Computing candidates Probabilities Warning: Deltaz: Deltaz computation failed for half range 5 samples Warning: Deltaz: Deltaz computation failed for half range 2 samples Error: Deltaz: Deltaz for candidate 0.000500 couldnt be calculated Info: Integrating 1 candidates proba. Info: CPdfCandidateszResult::Compute pdf peaks info (method=direct integration) Info: Saving stellar fitting results Info: Chisquare2Solve-result: extracting best redshift from PDF: z=-0.001000 Info: Chisquare2Solve-result: extracted best model: model=A_uka0v.dat Info: QSO solve enabled : no Info: Linemodel parameters: Info: -linetypefilter: no Info: -lineforcefilter: no Info: -fittingmethod: individual Info: -linewidthtype: combined Info: -instrumentresolution: 4300.00 Info: -velocity emission: 200.00 Info: -velocity absorption: 300.00 Info: -velocity fit: yes Info: -nsigmasupport: 8.0 Info: -em velocity fit min : 10.0 Info: -em velocity fit max : 400.0 Info: -em velocity fit step : 20.0 Info: -abs velocity fit min : 150.0 Info: -abs velocity fit max : 500.0 Info: -abs velocity fit step : 50.0 Info: -man velocity fit dz min : -6.00e-04 Info: -man velocity fit dz max : 6.00e-04 Info: -man velocity fit dz step : 1.00e-04 Info: -rigidity: tplshape Info: -tplratio_catalog: linecatalogs_tplshapes/linecatalogs_tplshape_ExtendedTemplatesJan2017v3_20170602_B14C_v6_emission Info: -tplratio_ismfit: no Info: -tplfit_priors_dirpath: Info: -tplratio_priors_betaA: 1.000000 Info: -tplratio_priors_betaTE: 1.000000 Info: -tplratio_priors_betaZ: 1.000000 Info: -linemodel offsets_catalog: linecatalogs_offsets/offsetsCatalogs_20170410_m150 Info: -continuumcomponent: tplfit Info: -tplfit_ismfit: yes Info: -tplfit_igmfit: yes Info: -continuum fit count: 1 Info: -tplfit_ignorelinesupport: no Info: -tplfit_secondpass-LC-fitting-method: -1 Info: -tplfit_priors_dirpath: Info: -tplfit_priors_betaA: 1.000000 Info: -tplfit_priors_betaTE: 1.000000 Info: -tplfit_priors_betaZ: 1.000000 Info: -continuumreestimation: no Info: -extremacount: 5 Info: -extremacount-firstpass B: 0 Info: -extrema cut proba-threshold: -1 Info: -first pass: Info: -largegridstep: 0.001000 Info: -fittingmethod: individual Info: -tplratio_ismfit: no Info: -multiplecontinuumfit_disable: yes Info: -second pass: Info: -skip second pass: no Info: -continuum fit method: retryall Info: -pdf-stronglinesprior: -1.000000e+00 Info: -pdf-hapriorstrength: -1.000000e+00 Info: -pdf-euclidNHaEmittersPriorStrength: -1.000000e+00 Info: -pdf-modelpriorzStrength: -1.000000e+00 Info: -pdf-combination: marg Info: -pdf-margAmpCorrection: no Info: -save-intermediate-chisquaretplshaperesults: 0 Info: Operator-Linemodel: FastFitLargeGrid enabled: 1781 redshifts will be calculated on the large grid (1782 initially) Info: Operator-Linemodel: TemplatesOrthogonalization enabled = 1 Info: Operator-Linemodel: Templates store prepared. Info: Operator-Linemodel: sourcesize init to: ss=0.10 Info: Operator-Linemodel: Tpl-ratios init Info: CatalogsTplShape - Loading tplshape catalog : linecatalogs_tplshapes/linecatalogs_tplshape_ExtendedTemplatesJan2017v3_20170602_B14C_v6_emission Info: CatalogsTplShape - Found 13 tplshaped velocities files Info: CatalogsTplShape - Found 0 tplshaped priors files Info: CatalogsTplShape - Found 0 tplshaped priorsPz files Info: CatalogsTplShape - Loaded 13 tplshaped catalogs Info: Operator-Linemodel: Lambda offsets init Info: CatalogsOffsets - Loading offsets catalog : linecatalogs_offsets/offsetsCatalogs_20170410_m150 Info: CatalogsOffsets - Loaded 1 lines offsets catalogs Info: Operator-Linemodel: initialized Info: Operator-Linemodel: continuum tpl fitting: sampling:log, step=1.00000e-03, min=1.00000e-02, max=4.99515e+00 Info: Operator-Linemodel: continuum tpl redshift list n=1785 Info: Operator-Linemodel: precomputing- with operator = chisquarelog Info: Operator-Linemodel: tplfit-precompute done in 4.5929e+00 sec Info: Operator-Linemodel: fitcontinuum_maxCount set to 1 Error: Operator-Linemodel: Negative amplitude found at z=0.00700: best continuum tpl ssp_100Myr_z008.dat, amplitude = 0.000000e+00 & error = 9.021289e-19 It seems that the problematic spectra are sky fibers, which have continuum level of almost zero. The parameter file and the pfsObject file have been already sent to Ali Allaoui .
|
| Comments |
| Comment by vlebrun [ 12/Nov/20 ] |
|
Actually the mean level is negative, which indeed provokes an error, as the code expects to fit a continuum with at least a null amplitude (within the error). I will investigate more deeply since the noise seems to be larger that flux, hence the spectrum should be treated as null, but extreme values around 1e-11 erg/cm2/s/AA could induce strange behaviors |