[INSTRM-1630] Apply INSTRM-907 TCP packet merging to the MPS command headers Created: 17/Jun/22  Updated: 02/Sep/23

Status: In Progress
Project: Instrument control development
Component/s: ics_mps_fpga
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: cloomis Assignee: cloomis
Resolution: Unresolved Votes: 0
Labels: EngRun
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to INSTRM-907 Controller FPGA code cannot handle co... Done

 Description   

In INSTRM-907 I added a buffering mechanism to try hard to merge commands split over multiple TCP packets (the Xilinx TCP stack treats the TCP stream as a stream of packets, not bytes). But I did not apply that to the command header. We are seeing split 8-byte header packets at Subaru, and need to protect against them.

Might be slightly trickier than the INSTRM-907 work but not much.



 Comments   
Comment by cloomis [ 17/Jun/22 ]

This change needs to be made because TCP never guarantees boundaries within the byte stream, and the FPGA cannot expect them.

But it is peculiar that the actors-ics VM is sending more such data than we saw at CIT or ASRD. One non-disruptive example is both repeatable and peculiar. When sending back-to-back HK commands for all 84 boards, in the second such command set the 43rd board request packet is split into a 6- and a 2-byte packet. This is after 641*N packets and more interestingly, when crossing the N*1024 per-stream TCP sequence number. Odd, but that did repeat three times.

I can check, but it may well be that these (legal and OK) TCP fragmentations always happen at the 1024*N boundaries, but that the INSTRM-907 work fixed them in nearly all cases. Some quirk about how we are commanding the cobras causes more headers to be fragmented.

I'll play with the VM OS sysctl knobs to see if I can perturb things. And can imagine a peephole on the FPGA command stream to insert dummy commands over the 1024*N byte boundaries.

Comment by cloomis [ 23/Jul/23 ]

Bump.

The last move in /data/MCS/20230722_009 ended with an FPGA ERROR 10. This is an error which I added when doing the INSTRM-907 work, and which indicates that the command parser cannot get a full header. So I think this is the same problem.

Comment by cloomis [ 23/Jul/23 ]

Also, register error code 10 in the fpgaProtocol decoder as "EMPTY COMMAND HEADER"

Comment by cloomis [ 23/Jul/23 ]

Code changes pushed. Need to get access to a Kintex 705 online at Subaru to test, along with a USB-connected PC (Linux or Windows) running some reasonable version of the Vivado SDK (just the command-line SDK needed). See my notes at https://github.com/Subaru-PFS/ics_mps_fpga/blob/master/INSTALL.md

Comment by cloomis [ 10/Aug/23 ]

One of the spare KC705s has been connected to a Linux box in Hilo, and the new code uploaded (from a remote development system). I still need to run some tests just with that system.

Yoshida, Hiroshige will connect the us-mcs computer to the real FPGA USB port, so we can update that one in the same way.

Generated at Sat Feb 10 16:37:44 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.