Damn. Happened again, on another channel (we earlier moved pump2 from channel 2 to 3, and just moved pump1 from 1 channel 1 to 4).

In any case, no errors. Only invalid because the commanded state is 1 but the current and pressure are 0.

2019-09-04 19:12:19.712Z cmds             20 CommandLink.py:121 > 1 3308 i ionPump1=1,7000,0.0018,35, 3.8e-06
2019-09-04 19:12:19.714Z cmds             20 CommandLink.py:121 > 1 3308 i ionPump1Errors=0x00,OK
2019-09-04 19:12:49.708Z cmds             20 CommandLink.py:121 > 1 3317 i ionPump1=1,7000,0.0015,34, 3.1e-06
2019-09-04 19:12:49.711Z cmds             20 CommandLink.py:121 > 1 3317 i ionPump1Errors=0x00,OK
2019-09-04 19:13:19.655Z cmds             20 CommandLink.py:121 > 1 3326 i ionPump1=1,7000,0.0013,34, 2.5e-06
2019-09-04 19:13:19.658Z cmds             20 CommandLink.py:121 > 1 3326 i ionPump1Errors=0x00,OK
2019-09-04 19:13:49.797Z cmds             20 CommandLink.py:121 > 1 3335 i ionPump1=1,7000,0.001,34, 1.7e-06
2019-09-04 19:13:49.800Z cmds             20 CommandLink.py:121 > 1 3335 i ionPump1Errors=0x00,OK
2019-09-04 19:14:19.712Z cmds             20 CommandLink.py:121 > 1 3344 i ionPump1=1,3100,0,34, 0
2019-09-04 19:14:19.715Z cmds             20 CommandLink.py:121 > 1 3344 i ionPump1Errors=0x00,OK
2019-09-04 19:14:49.705Z cmds             20 CommandLink.py:121 > 1 3353 i ionPump1=1,3100,0,33, 0
2019-09-04 19:14:49.708Z cmds             20 CommandLink.py:121 > 1 3353 i ionPump1Errors=0x00,OK
2019-09-04 19:15:03.001Z cmds             20 CommandLink.py:121 > 2 5327 i ionPump1=1,3100,0,33, 0
2019-09-04 19:15:03.002Z cmds             20 CommandLink.py:121 > 2 5327 i ionPump1Errors=0x00,OK

arnaud.lefur and I were just talking about issues like this in the alertActor, so I'll flesh this out a bit.

This is an example where simple alertsActor limits are not sufficient. We can either add alertsActor logic or ionpump controller logic.

Inside the alertsActor I think the way do do this would be to set a code callback on ionPump1Errors, and if it is 0x00/OK look at the ionPump1 keyword and check for state == 1 and pressure == 0. For slightly disgusting reasons: if the error keyword is not 0x00 we should use that, and the errors key comes just after the main keyword, so any logic has access to both keys.

FWIW there is no "ionpump error" STS datum, but we should make one, based on the numeric field. Also, the ionpump error key should grow a descriptive string field, as we have for the coolers and gate valve, etc.

But I think the fix should actually be made in the xcuActor, which should "create" an error bit in this instance.

I think we can just use the second (string) field as the alert description. So I guess create a STRING-only STS datum and use an (inverted) regex alert against "OK"? In any case this is back to being just an alertsActor ticket.

Known/tested bad values:

2019-09-04T21:15:13.438 xcu_b2 w ionpump2errors=0x8000,"Suspect live channel"
2019-09-04T21:18:25.529 xcu_b2 w ionpump2errors=0x0020,"Interlock cable"

vs.

2019-09-04T21:18:34.026 xcu_b2 i ionpump2errors=0x0000,OK

cloomis mentioned during the ICS+DRP/SPS telecon 2019-09-18 that this ticket is most likely already implemented. He will confirm (and close if appropriate) in the next day or so.

arnaud.lefur to review and close.

Implemented and tracked in STS as shown above from last cooldown.