https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-1111 Instrument control development <p>When reading ramps we need to stay ahead of the DAQ. I currently read from the DAQ and write to the FITS file in the same process, but any delay causes the DAQ to run out of buffer and for the reads to fail. A recent change in the readout sequence to minimize the effect of some electronic noise is making us hit the hardware limits.</p> <p>Add a process which handles the file writing. The tricky part is that we promise to generate keywords when we have flushed individual reads and the entire PFSA and PFSB files, and decoupling readouts from file writes will make that less trivial.</p> INSTRM-1111

Add file writing process to hxActors

Task Normal Done Done cloomis cloomis SPS Mon, 16 Nov 2020 02:21:57 +0000 Tue, 1 Mar 2022 21:29:20 +0000 Tue, 1 Mar 2022 21:29:20 +0000 ics_hxActor 0 1 <p>If we want this can also be added to the ccdActor/mcsActor.</p> <p>Can't say I like adding this kind of complexity (a process for the file writing and a thread for tracking progress from the process) in the core of data acquisition, but it does seem to work.</p> <p>Need to walk through all the new failure modes.</p> <p>This works in the sense of noticeably improving the performance of the reads from the SAM. But the two processes as a whole perform quite badly: either image transfers over the Queue are slow or something is bad about the i/o being done on the external process. Nothing obvious with the system.</p> <p>Will test with <tt>multiprocessing.shared_memory</tt></p> <p>Working cleanly with <tt>multiprocessing.shared_memory</tt>. That only comes with python 3.8 though. I am using a conda 3.9 environment and the actor is working fine. </p> <p>If that is not acceptable I can implement something similar using <tt>mmap</tt>. But would prefer to use a standard library routine.</p> <p>Merged at 20898d1, then transferred to ics_utils under <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1257" title="Move all common FITS, etc. routines to ics_utils" class="issue-link" data-issue-key="INSTRM-1257"><del>INSTRM-1257</del></a></p> Development Rank 0|02qpuc:i6001gr068fhw9qo4o0m38zi Sprint SM1PD-2021 A 12 SM1PD-2022 B Story Points 3.0