[REDMINE1D-129] [RM-7659] Gestion des pixels non numériques Created: 13/Jun/23  Updated: 23/Nov/23

Status: In Progress
Project: 1D Redmine
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: Redmine-Jira Migtation Assignee: Redmine-Jira Migtation
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Created on 2022-11-10 17:58:42 by Vincent Le Brun. % Done: 10

j'ai reçu un nouveau dataset de spectres venus de Fastspec et on dirait qu'ils contiennent des valeurs non numériques qu'on ne gère pas. A confirmer et voir si on veut gérer ce genre de chose
les spectres sont /net/CESAM/amazed/vlebrun/MAMBO/test_MAMBO_for_stacking/output_spectra_sel1DEEP-FORNAX_B30_R10/

 Comments   
Comment by Redmine-Jira Migtation [ 23/Nov/23 ]

Comment by Vincent Le Brun on 2022-11-10 17:59:25:
amazed se plante avant de lancer le job slurm avec ce message
/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/pyamazed/amazed.py:38: RuntimeWarning: divide by zero encountered in long_scalars
for _ in range(int(math.ceil(len(iterable)/nb_observations / bunch_size))):
Traceback (most recent call last):
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/pyamazed/amazed.py", line 259, in main
amazed(config)
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/pyamazed/amazed.py", line 99, in amazed
tasks = [(i, f'process_spectra_worker-

{i}', spectra)
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/pyamazed/amazed.py", line 99, in <listcomp>
tasks = [(i, f'process_spectra_worker-{i}

', spectra)
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/pyamazed/amazed.py", line 38, in bunch
for _ in range(int(math.ceil(len(iterable)/nb_observations / bunch_size))):
OverflowError: cannot convert float infinity to integer
Critical error: cannot convert float infinity to integer

Comment by Redmine-Jira Migtation [ 23/Nov/23 ]

Comment by Ali Allaoui on 2022-11-16 16:09:48:
j'ai pas réussi à reproduire ce problème , cf @/net/CESAM/amazed/aallaoui/amazed_workdir/deep_fornax_mambo@

Comment by Redmine-Jira Migtation [ 23/Nov/23 ]

Comment by Vincent Le Brun on 2022-11-16 16:38:12:
vizu -p 5002 /net/CESAM/amazed/aallaoui/amazed_workdir/deep_fornax_mambo/output/

{'import_dir': ['/net/CESAM/amazed/aallaoui/amazed_workdir/deep_fornax_mambo/output/'], 'reference': None, 'attributes': None, 'title': '', 'filter': None, 'mismatch_path': None, 'config_path': '/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/amazedWebServer/resources/report.json'}

Traceback (most recent call last):
File "/net/CESAM/amazed/venvs/amazed_0.36.1/bin/vizu", line 8, in <module>
sys.exit(run())
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/amazedWebServer/app.py", line 297, in run
session_id = mc.import_output_directory(config)
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/amazedWebServer/controller.py", line 41, in import_output_directory
session = Session(import_dir,
File "/net/CESAM/amazed/venvs/amazed_0.36.1/lib/python3.9/site-packages/AmazedOutputAnalyzer/session.py", line 64, in _init_
self.amazed_config["object_types"]["classification"] = "c"
KeyError: 'object_types'

Comment by Redmine-Jira Migtation [ 23/Nov/23 ]

Comment by Vincent Le Brun on 2023-08-29 15:48:37:
ça devrait être géré par les filtres ça ?

Comment by Redmine-Jira Migtation [ 23/Nov/23 ]

Comment by Ali Allaoui on 2023-09-19 08:27:31:
Le problème initial survenait à la préparation des bunches, donc soit un bug sur le bunch_size (=0?) ou sur le calcul du nombre d'observations (=0 ?) qui aurait pu entrainer cette exception (on divise par 0 -> inf -> on veut convertir inf en entier)
En tous cas rien à voir avec les pixels non numériques -> on ferme ?

Generated at Sat Feb 10 15:29:35 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.