https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-83 Instrument control development <p>After a while, a error occurs on the ccd status command and cause the temperature monitoring to stop.<br/> It's seem that the buffer isn't synchronised anymore.</p> <p>The only solution for now is to reboot the FEE. </p> <p>17:28:57.757250 .hub 0 cmds d CmdQueued=180320,1485883737.75,"client.alefur",2,"ccd_r1",5,"status"<br/> 17:28:57.777622 client.alefur 2 ccd_r1 w text='pathetic version string: unknown'<br/> 17:28:57.777876 client.alefur 2 ccd_r1 i version="unknown"<br/> 17:28:57.778100 client.alefur 2 ccd_r1 i text="exposure=None"<br/> 17:28:57.778327 client.alefur 2 ccd_r1 f text="command failed: RuntimeError(\"command echo mismatch. sent :'rt,FEE': rcvd :'264.87':\",) at /home/hostpfs/cpl/git/2016-11-30/ics_xcu_fpga/python/xcu_fpga/fee/feeControl.py:788"<br/> 17:28:57.778562 .hub 0 cmds d CmdDone=180320,"f</p> INSTRM-83

Echo mismatch from CCD temperature monitoring

Bug Major Done Done cloomis arnaud.lefur SM1 Tue, 31 Jan 2017 17:41:39 +0000 Thu, 12 Jul 2018 18:42:07 +0000 Thu, 12 Jul 2018 18:39:08 +0000 0 2 <p>Fee is doing okay so far, but this is worrysome.</p> <p>It happens two times yesterday, and again during the night.<br/> To fix it I did a power cycle and then a disconnect/connect controller for the first two times.<br/> But notice, that for the last time, I did only a connect/disconnect and it was working also.<br/> This means that the power cycling may not be required, and that it could be handled by software.</p> <p>Forget about that, I was using the wrong command.<br/> It's <b>ccd_r1 monitor controllers=temps period=0</b> </p> <p>Another bug which could be related or not.<br/> The monitoring isn't stopping when I ask for it. <br/> (I did a fee disconnect that's why you have this KeyError exception).</p> <p>09:52:50.199016 .hub 0 cmds d CmdIn="client.alefur","ccd_r1","monitor controllers=fee period=0"<br/> 09:52:50.199732 .hub 0 cmds d CmdQueued=1397466,1487325170.19,"client.alefur",26,"ccd_r1",95,"monitor controllers=fee period=0"<br/> 09:52:50.221279 client.alefur 26 ccd_r1 w text="adjusted fee loop to 0s"<br/> 09:52:50.221893 client.alefur 26 ccd_r1 : <br/> 09:52:50.222425 .hub 0 cmds d CmdDone=1397466,":"<br/> 09:52:59.733435 .ccd_r1 6187 ccd_r1 f text="command failed: KeyError('fee',) at python/ccdActor/main.py:108"<br/> 09:53:14.707476 .ccd_r1 6188 ccd_r1 f text="command failed: KeyError('fee',) at python/ccdActor/main.py:108"<br/> 09:53:29.721332 .ccd_r1 6189 ccd_r1 f text="command failed: KeyError('fee',) at python/ccdActor/main.py:108"</p> <p>It happened again this morning.<br/> A simple disconnect / connect of the fee controller was enough to fix it.</p> <p> <span class="image-wrap" style=""><a id="10602_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/10602/10602_bug_ccd2.png" title="bug_ccd2.png" file-preview-type="image" file-preview-id="10602" file-preview-title="bug_ccd2.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/10601/_thumb_10601.png" style="border: 0px solid black" /></a></span> <br/> Instead of commenting each time this bug happens I have analysed ccd logfile since we have put the science detector in late november.<br/> Here is a plot which show the bug frequency since then.</p> <p><span class="image-wrap" style=""><a id="10602_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/10602/10602_bug_ccd2.png" title="bug_ccd2.png" file-preview-type="image" file-preview-id="10602" file-preview-title="bug_ccd2.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/10602/_thumb_10602.png" style="border: 0px solid black" /></a></span> <br/> Note that there is 3 kind of bugs which occur more or less in same timing :</p> <ul> <li>"command failed: RuntimeError(\"command echo mismatch.",</li> <li>"command failed: ValueError('could not convert string to float",</li> <li>"command failed: SerialTimeoutException('Write timeout',)"</li> </ul> <p>The exposure runs in a separate thread from the command handler. So when the exposure timer ends and the device is read out two threads are communicating with the device. Added a lock around a complete command and response to mediate.</p> <p>Merged in 6785776, tag 70.4.4</p> Duplicate INSTRM-36 Relates INSTRM-84 INSTRM-190 INSTRM-226 Development Epic Link INSTRM-395 Rank 0|ii019f: