-
Type: Story
-
Status: Done (View Workflow)
-
Priority: Normal
-
Resolution: Done
-
Component/s: spt_operational_database
-
Labels:None
-
Story Points:4
In order to ensure regular processing of data from the summit (eg PIPE2D-706) at non-Subaru sites (e.g. in Princeton and Marseille), read-only access to data from the ‘live’ opDB at the summit is required. This access is needed to feed development and testing of both reductions themselves, and of any automated reduction scripts.
Since processing will be automated, we need direct access, not through a VPN or requiring 2FA.
Direct (read-only) access to the Hilo-based postgres port from some tiny list of external machines would be ideal. Barring that, chained streaming replication from Hilo to some other postgres instance would be fine, although quite a lot more work.
We can probably assume that the Hilo server will be just as good as the summit server. If streaming replication works to Hilo, as we expect, the servers will be effectively identical. See INSTRM-1145.
For the moment, we can work without direct access: dump-restore copies from any server suffices, and we can increase the cadence to meet initial needs (e.g. processing stability data from SM1). As we expect to use SuNSS to learn how to run PFS in regular operations we should put the final mechanism in place as soon as possible.
For developing and running the automated scripts it is valuable for all sites to have the same view of available data. Especially for debugging the reduction scripts, it would be useful to include the time sequence of the transactions as this allows us to reproduce the state of the opdb at any time. Without this full replication we will have to develop and debug on the mountain or in Hilo.
- blocks
-
PIPE2D-706 Regular processing of arcs, flats
- Done
- is blocked by
-
INSTRM-1145 Replication test for opDB using two test nodes in Hilo
- Done