[INSTRM-445] Provide 10TB disk at Subaru for MCS testing Created: 03/Aug/18  Updated: 07/Nov/19  Due: 31/Aug/18  Resolved: 07/Nov/19

Status: Done
Project: Instrument control development
Component/s: Summit infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: hassan Assignee: Yoshida, Hiroshige
Resolution: Done Votes: 0
Labels: MCS, Subaru, subaru-personnel
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As discussed during the PFS Servers and Infrastructure WG meeting 2018-07-25, a request by cloomis for a 10TB disk for installing the minimum software and data for the MCS testing in September 2018 needs to be made available.

Ideally this should be available by 2018-09-01 to allow Craig to install the necessary data.



 Comments   
Comment by hassan [ 03/Aug/18 ]

kyono: if this issue is not applicable to you please re-assign to the appropriate person or inform me or Tamura-san.

shimono: please move this issue to the appropriate JIRA project if this is not the correct one (or we need to create a new project).

Comment by Kiaina Schubert [ 09/Aug/18 ]

Hassan,

Discussion about some technical requirements for this request.

1.  This 10TB of additional disk space, not from current storage environment?

2.  IF yes to 1, which is prefferable,

  • Provision the best server R720(possible RGA1 replacement) with JBOD 6TB disk, with ZFS or FreeNAS as NFS server
  • the best server R720(possible RGA1 replacement) with 2 RAID1 OS SSD, 6 x 6TB RAID6 ~ 20TB usable space, nfs exported
Comment by cloomis [ 09/Aug/18 ]

This is purely a short-term resource, to cover us in the very unlikely case that the storage server fails before we implement a redundant storage system.

Someone at Subaru said that there is some general-purpose NFS storage available. That would be fine. Alternately we would put a pair of disks in the PFS NFS server and use that.

Comment by cloomis [ 09/Aug/18 ]

I will state that from my point of view, two of those 2xSSD + 6xHDD boxes provisioned as ZFS + NFS servers would be a good choice for a permanent system. Carve out a small intent cache from an SSD, and get someone to buy new data disks before real observing.

Yes, a properly engineered HA system would be ideal. I just don't know enough about that world in 2018 to say anything useful. The paired NFS servers, the paired VM servers, and some scripts is probably good enough. ~1 hour downtime max?

Comment by Kiaina Schubert [ 15/Aug/18 ]

Craig,

I been using a test server for the past year.  it has 6x4TB drives, running freenas.  Freenas is installed on USB stick.  I'm willing to put this into PFS for your usage. 

 

Kiaina

Comment by cloomis [ 15/Aug/18 ]

That certainly works for me, either as a short-term backup and or as a longer-term server if the hardware is otherwise appropriate. I would be interested in a small SSD-based ZIL if there is any chance of carving one out; we would need a ZIL for any long-term server, as well as newer/bigger drives (a fairly small expense). With a ZIL NFS writes are instantaneous...

Comment by Kiaina Schubert [ 15/Aug/18 ]

So basically, pulling 4TB and sticking a ssd in that slot, and configuring FreeNAS?  Can be done

 

Comment by cloomis [ 15/Aug/18 ]

Mmm, I'd rather have the 4+2 raid than any possible speedup. If you have no free slots, or an existing SSD then leave it as is. I think the minimum for a long-term system would be 2xSSD plus 6xHDD.

Comment by Kiaina Schubert [ 15/Aug/18 ]

Craig,

I know we promise two systems, can we dig up notes on  the purpose of the two systems again.

  1. replace old server - rfa1
  2. spare (additional) server NFS?

Was the expectation of the 10TB of NFS space part of #2?

Comment by cloomis [ 22/Aug/18 ]

Summary of conversation between CPL and CDM:

This ticket was just for providing emergency storage to allow us to recover from the (very unlikely) failure of the single RAID server. For that, we only need a small amount of NFS storage which can be hosted anywhere. The estimate of 10TB was (very) excessive. All we really need is:

  • /virt, which currently holds 250GB. 500GB would be ample.
  • /proddata, currently 100GB. Real data will be archived, 500GB-1TB would be ample until SM1.
  • /software, currently 5GB. 50GB ample
  • /home/xxx, currently 20GB. 100GB ample.

Frankly, 1TB would do, 2TB would be spacious. I'd prefer not to use the existing core PFS machines, if only because they have been provisioned via ansible, and adding this would be a bit of a hack.

In the longer term, we do need redundant storage, and hassan's https://sumire.pbworks.com/w/file/fetch/127236176/PFS-ICS-PRU030000-03_PFS_ICS_Hardware_Configuration_Report.pdf covers that.

One last note. CDM has a test FreeNAS/ZFS server (~15TB net, raid-z2 (4+2), can add small ZIL cache) which they would like to evaluate with PFS usage. I think this is a good idea. And yes, we could use that either as the backup we are discussing or even as the primary NFS server depending on how it evaluates.

Comment by cloomis [ 07/Nov/19 ]

No longer needed, I don't think.

Generated at Sat Feb 10 16:25:05 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.