[INFRA-247] Location for raw AMBRE data for F-stars Created: 11/Apr/19  Updated: 11/Apr/19  Resolved: 11/Apr/19

Status: Done
Project: Software Development Infrastructure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: hassan Assignee: rhl
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to INFRA-246 Location for F-star template data Won't Fix

 Description   

As well as a storage location for the derived F-star template data (INFRA-246), a storage location for the AMBRE ‘Warm’ library for relatively hot stars, which is the source of the template data, needs to be agreed on. Accessing these data from the original providers is not straightforward (the library author needs to be contacted).

The uncompressed volume of these data is about 430 GB. A subset of these data are needed for the generation of the template data, so in practice a few hundred GBs of data are required, uncompressed.

An alternative possibility is for NOAJ to store these raw data locally, in which case this issue is not longer relevant. Masayuki Tanaka to comment.

Reference AMBRE paper: https://arxiv.org/pdf/1205.2270.pdf

 Comments   
Comment by rhl [ 11/Apr/19 ]

Why do we need both in operations?

 

Comment by hassan [ 11/Apr/19 ]

We do not need the raw storage of AMBRE data for actual operations. If it is more appropriate, these data could be stored internally at NOAJ. Then this issue can be closed with no further action. Masayuki Tanaka to comment.

Comment by Masayuki Tanaka [ 11/Apr/19 ]

The original estimate of 430Gbytes was incorrect, sorry.  Each spectrum is 11Mbytes in the fits format.  There are 10575 spectra in the 'warm' (Teff>4000K) library that we have at NAOJ and the total volume is 107Gbytes.  If we restrict ourselves to 5500<Teff<8000K, for instance, the total will be about 65Gbytes.

The 2d pipeline does not need these 'raw' spectra.  If needed, I can put a compressed tarball with all the spectra somewhere accessible from anyone (likely an HSC machine).  Let me know.

Comment by hassan [ 11/Apr/19 ]

Following 2D DRP technical telecon today: agreed to close this ticket based on Masayuki Tanaka 's proposal of storing the raw data locally at NAOJ.

Generated at Sat Feb 10 16:50:54 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.