[INFRA-246] Location for F-star template data Created: 06/Apr/19 Updated: 13/Apr/19 Resolved: 13/Apr/19 |
|
| Status: | Won't Fix |
| Project: | Software Development Infrastructure |
| Component/s: | newitem |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Normal |
| Reporter: | hassan | Assignee: | yuki.moritani |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Story Points: | 2 | ||||||||||||||||||||
| Description |
|
In order to address The total size of that data is still TBC, but can vary between 30 and 300 GB. Takuji Yamashita to confirm. |
| Comments |
| Comment by rhl [ 06/Apr/19 ] |
|
Depending on how this data is used in the pipelines it may need to be visible as a path relative to the root of the raw data. This isn't a problem (as we can use symbolic links), but we should be aware. We will also need to make this data available at all PFS sites, and handle versioning it as our knowledge evolves. Those data volumes seem high to me, and I suspect that we can design a much more efficient format if necessary. |
| Comment by Masayuki Tanaka [ 11/Apr/19 ] |
|
Here is a revised estimate. If we store only the flux, each file (including extrapolation) will be about 4Mbytes in the fits format. The spectral sampling is 0.01A/pix, which is a very fine sampling and we do not need that. If we bin the spectra to, e.g., 0.1A/pix, which is still fine enough for our purpose, the data volume will be 1/10. We have not decided yet which type of stars to use, but if we limit ourselves to 5500<Teff[K]<8000K, which roughly corresponds to mid-G to early-A, there are ~6000 spectra. So, 4Mbytes * 0.1 * 6000 = 2.4Gbytes in total. |
| Comment by hassan [ 13/Apr/19 ] |
|
Following the 2D DRP technical telecon 2019-04-12: The total volume of the data is now relatively small. As these data need to be available at all locations where the pipeline is being run, we should only specify where in the Butler path these data are located. I will therefore close this ticket and create a new one for the Butler location. |
| Comment by hassan [ 13/Apr/19 ] |
|
Work will now be addressed under DAMD-51. |