The spectra currently used for flux standards in the simulator are from AMBRE, provided by Masayuki Tanaka.

Takuji Yamashita, could you provide the extrapolated version of these templates?

OK!

Should I make a branch for this on /drp_instdata/data/objects/fluxCal/ and upload the extrapolated templates there?

That would be great, thanks. Could you please upload only the spectra corresponding to the ones that already exist (we're trying to keep the size of the package from becoming overwhelmingly large) and update the README.txt file appropriately?

I have placed the extrapolated spectra and an updated README.txt in a new branch. These spectra were generated in the ticket of PIPE2D-364. The data format was changed from the original version. Only flux density in nJy is stored, you need to use FITST headers for wavelength solution, and the pixel scale is 0.01 nm/pix now. If you have any questions, please feel free to contact me.

I got a warning from git:

Encountered 6 file(s) that should have been pointers, but weren't:
	data/objects/fluxCal/p6500_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits
	data/objects/fluxCal/p6500_g+4.0_m0.0_t01_z-1.00_a+0.00.AMBRE_Extp.fits
	data/objects/fluxCal/p7000_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits
	data/objects/fluxCal/p7000_g+4.0_m0.0_t01_z-1.00_a+0.00.AMBRE_Extp.fits
	data/objects/fluxCal/p7500_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits
	data/objects/fluxCal/p7500_g+4.0_m0.0_t01_z-1.00_a+0.00.AMBRE_Extp.fits

Could you please check that you're using git lfs?

Adding files with a different format will require coordinated changes in drp_instmodel.

The spectrum appears to be discontinuous at the red end: the continuum doesn't match.

>>> import astropy.io.fits
>>> ff = astropy.io.fits.open("data/objects/fluxCal/p6500_g+4.0_m0.0_t01_z+0.00_a+0.00.AMBRE_Extp.fits")
>>> flux = ff[1].data.Flux
>>> import numpy as np
>>> wavelength = np.arange(len(flux))*0.01 + 300
>>> import matplotlib.pyplot as plt
>>> plt.plot(wavelength, flux, 'k-')
>>> plt.show()

I did not use git lfs. I do again.

According to DAMD-50, the discussion on the data format is not fixed yet. We need to decide that,  Masayuki Tanaka and hassan?

I made those extrapolation for the spectra which were convolved with LSF (Gaussian), while the stored spectra at 1200nm are ones before the LSF convolution. Please use them after the LSF convolution on your side. I can send the LSF-convolved spectra but I thought the unconvolved data is better because you can apply your LSF.

The spectrum that was convolved by an LSF (a Gaussian with FWHM = 2.4 \AA) is overplotted. A small "jump" remains at 1200 nm. This jump is smaller than 1% which is our target accuracy of flux calibration. I believe this is not a serious problem for flux calibration, but it is nice if we can resolve this. So, please let me add this issue to my development item list after my first loop of development. 

A function fitting for extrapolation was done on a spectrum with small absorption lines and therefore the extrapolation function is an "average" including small absorption lines and is a bit lower than the continuum. This is why we find the jump there. 

The FITS files were uploaded to github again using git lfs. Please check it. 

The FITS files are still present in git, from the previous commit. This makes the git repo larger than it should be (this is the whole reason for git lfs). I'll fix it, as it requires editing history.

I've fixed the commits on the branch. If you have any problems on your local branch, please git fetch && git reset --hard origin/tickets/PIPE2D-703 (be warned that this will delete any local changes).

I've updated the simulator code that reads these files, and also the pipeline code that uses them.

We'll also want to regenerate the weekly raw dataset before this merges. hassan, are you aware of any other coming changes that will require regenerating the weekly, so we can coordinate them?

To my understanding price, there are no other upcoming changes that require regenerating the weekly. DAMD-100, DAMD-101 and DAMD-105 do change PfsConfig and PfsDesign files, but as we agreed I will ensure that the previous versions of those files can be read, for a certain period (ie., when we schedule a new weekly raw dataset generation).

When are we getting Sean's new object lists?  And when are we going to use different target lists for brn and bmn, that's a structural change to the weekly.  That'll require DAMD-105, won't it?

I've agreed with Paul offline about 30 mins ago that we will regenerate the weekly after DAMD-105, DAMD-100, DAMD-101 are implemented in any case, given that I should be able to complete them in the next few days.

I believe this is ready to merge, and we should include it in the new generation of sims.

Paul has reviewed the changes.

Merged.