[PIPE1D-11] pytest failed for release 0.12.0 Created: 11/Jun/19  Updated: 03/Jul/20  Resolved: 03/Jul/20

Status: Done
Project: DRP 1D pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: Kiyoto Yabe Assignee: Pierre-Yves CHABAUD
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I'm trying to install the latest version on a debian server. drp_1d seems to be OK with the test passed, but I got failures below during the pytest process for drp_1dpipe. This might happen in the older version, but I don't remember what was wrong. 

 

============================================================================================= test session starts =============================================================================================
platform linux – Python 3.5.3, pytest-4.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /home/pfs/src/drp_1dpipe, inifile:
collected 8 items

drp_1dpipe/tests/test_io.py . [ 12%]
drp_1dpipe/tests/test_merge_results.py . [ 25%]
drp_1dpipe/tests/test_pre_process.py F. [ 50%]
drp_1dpipe/tests/test_utils.py ..F. [100%]

================================================================================================== FAILURES ===================================================================================================
__________________________________________________________________________________________________ test_run ___________________________________________________________________________________________________

def test_run():
"""
The "test_run" function.

This function test the "run" function of "pre_process.py" module.
"""

workdir = TemporaryDirectory()
args = FakeArgs(workdir.name)

fits_file = []
for i in range(9):
with open(normpath(args.spectra_path, '{}.fits'.format), 'w') as f:
fits_file.append(os.path.basename(f.name))

> result_run = run(args)

drp_1dpipe/tests/test_pre_process.py:61:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = <drp_1dpipe.tests.test_pre_process.FakeArgs object at 0x7f278421c2b0>

def run(args):
"""
Prepare workdir for process_spectra.

This function creates a json file containing a list of list of spectra.

:param args: parsed arguments of the program.
:return: 0 on success
"""

  1. initialize logger
    init_logger("pre_process", args.logdir, args.loglevel)

> spectra_dir = normpath(args.workdir, args.spectra_dir)
E AttributeError: 'FakeArgs' object has no attribute 'spectra_dir'

drp_1dpipe/pre_process/pre_process.py:67: AttributeError
_____________________________________________________________________________________________ test_args_from_file _____________________________________________________________________________________________

def test_args_from_file():
"""
The "test_args_from_file" function.

This function tests feature of retrieving argument value from
configuration file
"""
fp1 = tempfile.NamedTemporaryFile()
conf_file = fp1.name
with open(conf_file, 'w') as cf:
cf.write('arg1 = 4\n')
cf.write('arg2 = foo2 foo2\n')
cf.write('arg3 = foo3 # test\n')
cf.write('arg4 = # foo4\n')
cf.write('arg5 # = foo5\n')
cf.write('#arg6 = foo6')
cf.write('arg7 arg7 = foo7\n')

class MyCls():
arg1 = "2"

args = MyCls()
> get_args_from_file(conf_file, args)
E TypeError: get_args_from_file() takes 1 positional argument but 2 were given

drp_1dpipe/tests/test_utils.py:59: TypeError
============================================================================================== warnings summary ===============================================================================================
/home/pfs/.local/lib/python3.5/site-packages/pyfits/_init_.py:22
/home/pfs/.local/lib/python3.5/site-packages/pyfits/_init_.py:22: PyFITSDeprecationWarning: PyFITS is deprecated, please use astropy.io.fits
PyFITSDeprecationWarning) # noqa

– Docs: https://docs.pytest.org/en/latest/warnings.html
=============================================================================== 2 failed, 6 passed, 1 warnings in 0.55 seconds ================================================================================



 Comments   
Comment by Kiyoto Yabe [ 03/Jul/19 ]

Thank you for your fixing that. Now I don't see any errors in pytest:

============================================================ test session starts ============================================================
platform linux – Python 3.5.3, pytest-5.0.0, py-1.8.0, pluggy-0.12.0
rootdir: /home/pfs/src/drp_1dpipe
collected 8 items

drp_1dpipe/tests/test_io.py . [ 12%]
drp_1dpipe/tests/test_merge_results.py . [ 25%]
drp_1dpipe/tests/test_pre_process.py .. [ 50%]
drp_1dpipe/tests/test_utils.py .... [100%]

============================================================= warnings summary ==============================================================
/home/pfs/.local/lib/python3.5/site-packages/pyfits/_init_.py:22
/home/pfs/.local/lib/python3.5/site-packages/pyfits/_init_.py:22: PyFITSDeprecationWarning: PyFITS is deprecated, please use astropy.io.fits
PyFITSDeprecationWarning) # noqa

– Docs: https://docs.pytest.org/en/latest/warnings.html
=================================================== 8 passed, 1 warnings in 0.50 seconds ====================================================

Comment by Kiyoto Yabe [ 03/Jul/19 ]

But, on the other hand, I got the following error during processing using example data. Is this because of using python3.5? If necessary, I'll file another ticket.

 

=================================

Traceback (most recent call last):
File "/usr/local/bin/process_spectra", line 11, in <module>
load_entry_point('drp-1dpipe==0.12.1', 'console_scripts', 'process_spectra')()
File "/home/pfs/.local/lib/python3.5/site-packages/pkg_resources/_init_.py", line 489, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/home/pfs/.local/lib/python3.5/site-packages/pkg_resources/_init_.py", line 2843, in load_entry_point
return ep.load()
File "/home/pfs/.local/lib/python3.5/site-packages/pkg_resources/_init_.py", line 2434, in load
return self.resolve()
File "/home/pfs/.local/lib/python3.5/site-packages/pkg_resources/_init_.py", line 2440, in resolve
module = _import_(self.module_name, fromlist=['__name__'], level=0)
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/process_spectra/process_spectra.py", line 159
f'unable to read parameter file : {e}, using defaults')
^
SyntaxError: invalid syntax
— Logging error —
Traceback (most recent call last):
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/scheduler.py", line 128, in run
'output-dir': 'output-'})
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/local.py", line 82, in parallel
'{}'.format([f.result() for f in futures]))
Exception: A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/pfs/wrk/drp1d/test/test20190703/spectralist_3mo7ofm2', '--loglevel=WARNING', '--logdir=logs', '--lineflux=on', '--spectra-dir=spectra', '--workdir=/home/pfs/wrk/drp1d/test/test20190703', '--output-dir=output-0'], returncode=1)]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.5/logging/_init_.py", line 981, in emit
msg = self.format(record)
File "/usr/lib/python3.5/logging/_init_.py", line 831, in format
return fmt.format(record)
File "/usr/lib/python3.5/logging/_init_.py", line 568, in format
record.message = record.getMessage()
File "/usr/lib/python3.5/logging/_init_.py", line 331, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "/usr/local/bin/drp_1dpipe", line 11, in <module>
load_entry_point('drp-1dpipe==0.12.1', 'console_scripts', 'drp_1dpipe')()
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/scheduler.py", line 60, in main
return run(args)
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/scheduler.py", line 130, in run
logger.log(logging.ERROR, 'Error in process_spectra:', e)
Message: 'Error in process_spectra:'
Arguments: (Exception("A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/pfs/wrk/drp1d/test/test20190703/spectralist_3mo7ofm2', '--loglevel=WARNING', '--logdir=logs', '--lineflux=on', '--spectra-dir=spectra', '--workdir=/home/pfs/wrk/drp1d/test/test20190703', '--output-dir=output-0'], returncode=1)]",),)
— Logging error —
Traceback (most recent call last):
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/scheduler.py", line 128, in run
'output-dir': 'output-'})
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/local.py", line 82, in parallel
'{}'.format([f.result() for f in futures]))
Exception: A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/pfs/wrk/drp1d/test/test20190703/spectralist_3mo7ofm2', '--loglevel=WARNING', '--logdir=logs', '--lineflux=on', '--spectra-dir=spectra', '--workdir=/home/pfs/wrk/drp1d/test/test20190703', '--output-dir=output-0'], returncode=1)]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.5/logging/_init_.py", line 981, in emit
msg = self.format(record)
File "/usr/lib/python3.5/logging/_init_.py", line 831, in format
return fmt.format(record)
File "/usr/lib/python3.5/logging/_init_.py", line 568, in format
record.message = record.getMessage()
File "/usr/lib/python3.5/logging/_init_.py", line 331, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "/usr/local/bin/drp_1dpipe", line 11, in <module>
load_entry_point('drp-1dpipe==0.12.1', 'console_scripts', 'drp_1dpipe')()
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/scheduler.py", line 60, in main
return run(args)
File "/home/pfs/.local/lib/python3.5/site-packages/drp_1dpipe/scheduler/scheduler.py", line 130, in run
logger.log(logging.ERROR, 'Error in process_spectra:', e)
Message: 'Error in process_spectra:'
Arguments: (Exception("A task returned non-zero value. [CompletedProcess(args=['process_spectra', '--spectra-listfile=/home/pfs/wrk/drp1d/test/test20190703/spectralist_3mo7ofm2', '--loglevel=WARNING', '--logdir=logs', '--lineflux=on', '--spectra-dir=spectra', '--workdir=/home/pfs/wrk/drp1d/test/test20190703', '--output-dir=output-0'], returncode=1)]",),)

Generated at Sat Feb 10 15:35:27 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.