[INFRA-246] Location for F-star template data Created: 06/Apr/19  Updated: 13/Apr/19  Resolved: 13/Apr/19

Status: Won't Fix
Project: Software Development Infrastructure
Component/s: newitem
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: hassan Assignee: yuki.moritani
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks SIM2D-113 Provide F-star spectra Done
Relates
relates to INFRA-247 Location for raw AMBRE data for F-stars Done
relates to DAMD-51 Theoretical spectra for flux calibrat... Open
Story Points: 2

 Description   

In order to address SIM2D-113, a physical location to store and share F-star data needs to be discussed and agreed between the PFS Project Office and Princeton.

The total size of that data is still TBC, but can vary between 30 and 300 GB. Takuji Yamashita to confirm.



 Comments   
Comment by rhl [ 06/Apr/19 ]

Depending on how this data is used in the pipelines it may need to be visible as a path relative to the root of the raw data.  This isn't a problem (as we can use symbolic links), but we should be aware.  We will also need to make this data available at all PFS sites, and handle versioning it as our knowledge evolves.

Those data volumes seem high to me, and I suspect that we can design a much more efficient format if necessary.

Comment by Masayuki Tanaka [ 11/Apr/19 ]

Here is a revised estimate.  If we store only the flux, each file (including extrapolation) will be about 4Mbytes in the fits format.  The spectral sampling is 0.01A/pix, which is a very fine sampling and we do not need that.  If we bin the spectra to, e.g., 0.1A/pix, which is still fine enough for our purpose, the data volume will be 1/10.  We have not decided yet which type of stars to use, but if we limit ourselves to 5500<Teff[K]<8000K, which roughly corresponds to mid-G to early-A, there are ~6000 spectra.  So, 4Mbytes * 0.1 * 6000 = 2.4Gbytes in total.

Comment by hassan [ 13/Apr/19 ]

Following the 2D DRP technical telecon 2019-04-12: The total volume of the data is now relatively small. As these data need to be available at all locations where the pipeline is being run, we should only specify where in the Butler path these data are located.

I will therefore close this ticket and create a new one for the Butler location.

Comment by hassan [ 13/Apr/19 ]

Work will now be addressed under DAMD-51.

Generated at Sat Feb 10 16:50:53 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.