[INSTRM-1282] Possible command timeouts from FEE Created: 21/May/21  Updated: 07/Jun/21  Resolved: 07/Jun/21

Status: Won't Fix
Project: Instrument control development
Component/s: ics_xcu_fee, ics_xcu_fpga
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: cloomis Assignee: cloomis
Resolution: Won't Fix Votes: 0
Labels: SPS
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates INSTRM-786 ccd_r1 wipe failed Won't Fix
Relates
relates to INSTRM-786 ccd_r1 wipe failed Won't Fix
relates to INSTRM-1284 Handle FEE mode errors individually Done
Sprint: SM1PD-2021 A10

 Description   

The ccd actor sometimes fails to get a response from a FEE command. Cannot tell which side is the problem. Almost always the same command.

Cannot trivially distinguish between cases where the FEE/link misbehaved and when it might have been disconnected or otherwise worked on, so the logs at LAM and (especially) JHU are hard to make use of.

At Subaru, looks like ~4 real instances for r1 and ~4 for b1 since 2020-01-01.

Here is an example of one good, one failed command from this morning. Both the most command command to fail on (by far): a mode change, which switches voltages levels for all the clocks and bias voltages. Usually takes ~0.1s, so 2s is awry.

2021-05-21 03:55:50.684Z root             10 feeControl.py:904 sending command :'~lp,idle\n':
2021-05-21 03:55:50.688Z root             10 feeControl.py:962 received :~lp,idle:
2021-05-21 03:55:50.745Z root             10 feeControl.py:962 received :SUCCESS:
2021-05-21 03:56:03.487Z cmds             20 Actor.py:507 new cmd: wipe
2021-05-21 03:56:03.488Z root             10 feeControl.py:904 sending command :'~lp,idle\n':
2021-05-21 03:56:03.492Z root             10 feeControl.py:962 received :~lp,idle:
2021-05-21 03:56:05.494Z root             30 feeControl.py:953 pyserial device read(1) timed out
2021-05-21 03:56:05.495Z root             10 feeControl.py:962 received ::


 Comments   
Comment by cloomis [ 07/Jun/21 ]

Closing this; leaving INSTRM-786.

Generated at Sat Feb 10 16:33:46 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.