Uploaded image for project: 'DRP 1D pipeline'
  1. DRP 1D pipeline
  2. PIPE1D-12

drp_1dpipe does not read pfsObject** file created by PFS spectrum simulator

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Done (View Workflow)
    • Priority: Normal
    • Resolution: Done
    • Component/s: None
    • Labels:
      None

      Description

      I have run the drp_1dpipe on a pfsObject*** file, which is an output of the PFS spectrum simulator (Version 1.1). I've got following error messages: 

       

      ---------------------------------------------------------------------------------

      (base) miho@thomas:~/PFS/pipeline/GAtest$ drp_1dpipe --workdir /home/miho/PFS/pipeline/GAtest
      Can't load spectrum : "Key 'wavelength' does not exist."
      Can't load spectrum : "Key 'wavelength' does not exist."
      Traceback (most recent call last):
      File "/home/miho/anaconda3/bin/process_spectra", line 11, in <module>
      load_entry_point('drp-1dpipe==0.12.1', 'console_scripts', 'process_spectra')()
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/process_spectra/process_spectra.py", line 96, in main
      return run(args)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/process_spectra/process_spectra.py", line 279, in run
      amazed(args)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/process_spectra/process_spectra.py", line 260, in amazed
      args.lineflux in ['only', 'on'])
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/process_spectra/results.py", line 50, in _init_
      self._read_lambda_ranges()
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/process_spectra/results.py", line 107, in _read_lambda_ranges
      hdul = fits.open(path)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 151, in fitsopen
      lazy_load_hdus, **kwargs)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 390, in fromfile
      lazy_load_hdus=lazy_load_hdus, **kwargs)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 1039, in _readfrom
      fileobj = _File(fileobj, mode=mode, memmap=memmap, cache=cache)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/utils/decorators.py", line 503, in wrapper
      return function(*args, **kwargs)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/io/fits/file.py", line 178, in _init_
      self._open_filename(fileobj, mode, overwrite)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/io/fits/file.py", line 555, in _open_filename
      self._file = fileobj_open(self.name, IO_FITS_MODES[mode])
      File "/home/miho/anaconda3/lib/python3.7/site-packages/astropy/io/fits/util.py", line 388, in fileobj_open
      return open(filename, mode, buffering=0)
      FileNotFoundError: [Errno 2] No such file or directory: '/home/miho/PFS/pipeline/GAtest/spectra/pfsObject-1234-57011-0,0-000a083d-000002.fits'
      — Logging error —
      Traceback (most recent call last):
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/scheduler.py", line 128, in run
      'output-dir': 'output-'})
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/local.py", line 82, in parallel
      '{}'.format([f.result() for f in futures]))
      Exception: A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/miho/PFS/pipeline/GAtest/spectralist_4fcgkdvu', '--workdir=/home/miho/PFS/pipeline/GAtest', '--logdir=/home/miho/PFS/pipeline/GAtest/logdir', '--loglevel=WARNING', '--lineflux=on', '--spectra-dir=spectra', '--output-dir=output-0'], returncode=1)]

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 1034, in emit
      msg = self.format(record)
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 880, in format
      return fmt.format(record)
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 619, in format
      record.message = record.getMessage()
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 380, in getMessage
      msg = msg % self.args
      TypeError: not all arguments converted during string formatting
      Call stack:
      File "/home/miho/anaconda3/bin/drp_1dpipe", line 11, in <module>
      load_entry_point('drp-1dpipe==0.12.1', 'console_scripts', 'drp_1dpipe')()
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/scheduler.py", line 60, in main
      return run(args)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/scheduler.py", line 130, in run
      logger.log(logging.ERROR, 'Error in process_spectra:', e)
      Message: 'Error in process_spectra:'
      Arguments: (Exception("A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/miho/PFS/pipeline/GAtest/spectralist_4fcgkdvu', '--workdir=/home/miho/PFS/pipeline/GAtest', '--logdir=/home/miho/PFS/pipeline/GAtest/logdir', '--loglevel=WARNING', '--lineflux=on', '--spectra-dir=spectra', '--output-dir=output-0'], returncode=1)]"),)
      — Logging error —
      Traceback (most recent call last):
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/scheduler.py", line 128, in run
      'output-dir': 'output-'})
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/local.py", line 82, in parallel
      '{}'.format([f.result() for f in futures]))
      Exception: A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/miho/PFS/pipeline/GAtest/spectralist_4fcgkdvu', '--workdir=/home/miho/PFS/pipeline/GAtest', '--logdir=/home/miho/PFS/pipeline/GAtest/logdir', '--loglevel=WARNING', '--lineflux=on', '--spectra-dir=spectra', '--output-dir=output-0'], returncode=1)]

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 1034, in emit
      msg = self.format(record)
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 880, in format
      return fmt.format(record)
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 619, in format
      record.message = record.getMessage()
      File "/home/miho/anaconda3/lib/python3.7/logging/_init_.py", line 380, in getMessage
      msg = msg % self.args
      TypeError: not all arguments converted during string formatting
      Call stack:
      File "/home/miho/anaconda3/bin/drp_1dpipe", line 11, in <module>
      load_entry_point('drp-1dpipe==0.12.1', 'console_scripts', 'drp_1dpipe')()
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/scheduler.py", line 60, in main
      return run(args)
      File "/home/miho/anaconda3/lib/python3.7/site-packages/drp_1dpipe-0.12.1-py3.7.egg/drp_1dpipe/scheduler/scheduler.py", line 130, in run
      logger.log(logging.ERROR, 'Error in process_spectra:', e)
      Message: 'Error in process_spectra:'
      Arguments: (Exception("A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/miho/PFS/pipeline/GAtest/spectralist_4fcgkdvu', '--workdir=/home/miho/PFS/pipeline/GAtest', '--logdir=/home/miho/PFS/pipeline/GAtest/logdir', '--loglevel=WARNING', '--lineflux=on', '--spectra-dir=spectra', '--output-dir=output-0'], returncode=1)]"),)

       

      ------------------------------------------------------------------------------

      According to the first line of the messages, it seems that the input pfsObject*** file 

      does not have a fits header keyword 'wavelength'.

      The drp_1dpipe code runs well with no errors for input spectra included in "pfs_datapack_sp8_v2.0". 

      I attach the input spectrum I'm trying to use (output from the PFS spectrum simulator Ver.1.1). 

       

        Attachments

          Activity

            People

            • Assignee:
              pychabau Pierre-Yves CHABAUD
              Reporter:
              ishigaki ishigaki
              Reviewers:
              ishigaki
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: