s/w tasks for 2019-06 JHU-LAM visit

Just a ticket to put all the s/w prep and changes required for the hardware work at LAM in mid 2019-06. Add to this Description if necessary: this should be a valid list.

For new interlock board and gauges:

• b1 and r1 (and r2?) BIOSes need to be updated to split first serial port into two. This requires working serial console access to all BEEs.
• Update to (at least) xcuActor 1.9.3. This will include config changes to support the turbo moving to /dev/ttyS3, as well as for the new gauges and interlock boards.
• Modify the ics_ccdActor configs to move the fee to /dev/ttyS2
• Update ics_actorkeys: there are new keywords for interlock status and pressures.

For new temperature board:

• Install ics_xcu_SMB on the temperature board pis. Commands and keywords unchanged. The Pis are now standalone (no /software), so no software needs to be installed at LAM. (DONE)
• At JHU: the boards should be delivered this week and Josh needs to provision and test them. (DONE, including two spares for LAM)

In general:

• Update dnsmasq.d: new temperature boards, gauge maintenance addresses. I'm putting the MAC changes in a macs-id/forLAM.conf file. To support the two spare temps boards (i.e. with known MAC addresses but unassigned IP addresses, the dnsmasq-site.lam config needs to have known and !known dhcp-ranges, like in the IDG and Subaru site configs. Just grab the stanza from dnsmasq-site.idg.
• Switch to /software/ics_launch, probably. In any case, drop the turbo TCP shim.
• Make sure image files end up in /data/pfs/ (for SM1 pre-ship)
• Rename existing archiver database, create new one (this for SM1 pre-ship)

Schedule

Pre-ship:

1. make SMB OS image (DONE, running at JHU)
2. tag SMB program (DONE, running at JHU. 1.2.0 but might be updated.)
3. get BEE console access (DONE LAM)
4. put the new PCM firmware image someplace nice. Sorry, has simply been put in the LAM /software/mhs/.

Before power down:

1. install tron_actorcore:2.0.3 and ics_xcuActor:1.9.5 – for the PCM update.
2. update the PCM firmware, to support the new gauges. This reboots the BEEs, so should be done at the last minute. Would be safe to do after the work, but is strange enough to want to do separately. Specifically:

     setup ics_xcuActor
     oneCmd.py xcu_b1 pcm raw=~gr # Will return _some_ version.
     PCM_bootloader.py --debug --cam=b1 --hexfile=/software/mhs/PCM_main-20190517_02.hex
     oneCmd.py xcu_b1 pcm raw=~gr # Should show newer version.
     

3. reconfigure the BEE BIOSes to get four serial ports. Specifically:

     # In one terminal:
     miniterm.py --raw socket://moxa-rack-1.pfs:4004
     # At LAM (only) b1 is on port 4004 and r1 is on 4003
   
     # In some other other terminal:
     ssh root@bee-b1 shutdown -r now
   
     # Start hitting F4 in miniterm window. 
     # Keep hitting it until a BIOS screen appears.
     
     # Under Advanced/Serial Port, configure CN7 for two ports (like CN8).
     # Under Advanced/RTD Specific, Disable aDIO IRQ
     # Under Exit, Save changes and exit.
     

4. Change the way PFS programs are started on the BEEs. Specifically:

• Remove anything interesting from /etc/rc.local. Leave only exit 0.
• For both the root and pfs-data users, login to BEE and crontab -e. At the bottom, add two lines:

  SHELL=/bin/bash
  @reboot /software/ics_launch/bin/launchFor.sh >> /tmp/${USER}boot.txt 2>&1
  

  Note that at this point the BEEs will not be useful until /software/ics_launch has been installed.

During power down for the hardware updates (all at LAM):

1. update ics_dnsmasq to support new temp boards. Specifically, incorporate the new host MACs listed in macs-id/forLAM.conf and add the known and !known address ranges.
2. Install ics_xcu_rtdADIO:1.1.1
3. install new tron_actorcore, ics_xcuActor, ics_xcu_fpga, ics_ccdActor.
4. install /software/ics_launch.

After restart, before pumping & cooling:

1. xcu and ccd programs should autostart from /software/ics_launch.
2. ping temps-[br]1 after powering them on. Sanity check.
3. temps status, turbo status, gauge status, fee status should all work, showing that the new BEE serial port mappings, the new xcu and ccd device configs, and the new temperature boards and gauges work.
4. Test interlock boards and sensors. Document is in https://github.com/Subaru-PFS/ics_xcuActor/blob/master/docs/PFS-SPS-PRU020100_Pumping_Control.rst