[PIPE2D-918] Provide flux data for calibration lamps and PFI for JEG Created: 22/Oct/21  Updated: 06/Dec/21  Due: 01/Dec/21  Resolved: 01/Dec/21

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: ncaplar Assignee: ncaplar
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File Argon_blue_spectrum.csv     PNG File Argon_red.png     File Argon_red_spectrum.csv     File bright_measured_lines_Argon_red.csv     File bright_measured_lines_Krypton_red.csv     File bright_measured_lines_Neon_blue.csv     File bright_measured_lines_Neon_red.csv     PNG File continuum_2d_extraction.png     File Continuum_blue_spectrum.csv     PNG File Continuum_Jim_data.png     PNG File Continuum_Jim_ErinDec2_data.png     PNG File Continuum_Jim_ErinDec2_data.png     PNG File continuum.png     File Continuum_red_spectrum.csv     Text File Jeg_Spectrum.txt     PNG File Krypton_red.png     File Krypton_red_spectrum.csv     PNG File Neon_blue.png     File Neon_blue_spectrum.csv     PNG File Neon_red.png     File Neon_red_spectrum.csv    
Issue Links:
Blocks
is blocked by PIPE2D-924 Create calibs and process recent data Done
is blocked by PIPE2D-933 Fix normalisation of extracted spectra Done
Relates
relates to INSTRM-1438 pfilamps controlled exposure time doe... Open
relates to PIPE2D-902 Measure the flux of calibration lamps... Done
relates to PIPE2D-911 Compare flux in observatiosn of sky w... Done
relates to INSTRM-1470 Blue/Red flux ratio lower than expected Open
Story Points: 2
Sprint: 2DDRP-2021 A12

 Description   

In the email from October 18, JEG asked me to provide him with the following data:

1. rates (electrons/sec or ADU/sec and an assumed gain at a small set of
wavelengths for the continuum lamp. I know these data exist,
because you showed them during the commissioning phonecons.

2. The same for the line lamps; rates for an identified line or, if
possible, a small set of identified lines.

3. This one may be harder. For the best (highest but not saturated)
exposure for each of the line lamps, could you send me an ASCII spectrum? Just wavelength and counts (ADU or electrons) for blue and red? Only the bias removed if you can do that

-------

Notebook at:

 /home/ncaplar/Pipeline_notebooks/PIPE2D-918 (flux question from JEG).ipynb


 Comments   
Comment by ncaplar [ 11/Nov/21 ]

For all of these figures and data, I have extracted data from 15 seconds exposure taken during the September run. The analysis has been done by extracting data from fiber 73. This fiber has been chosen for being on 90th percentile of brightness out of all fibers - so we are talking about one of the brightest fibers available. Note that these all results are in counts. For electron conversion, gain for the instrument is approx 1.22.

1. Requested data is specified in Continuum_red_spectrum.csv and Continuum_blue_spectrum.csv . It gives the full spectrum of continuum lamp, separately for blue and red lamp. The rates are given per pixel, per second - i.e., I have taken fluxes at a given wavelength as given by the pipeline and divided them by 3.22. Continuum reduction is currently not working very well - detectorMap is not adjusted for continuum observations. I have looked at the traces that the pipeline is using ( ) and those are not perfect, but reasonable.

2. bright_measured_lines_Argon_red.csv, bright_measured_lines_Neon_red.csv, bright_measured_lines_Krypton_red.csv and bright_measured_lines_Neon_blue.csv give 12 brightest lines as measured in the data. The flux values are amount of counts in the full psf, in a single second (NOT IN A SINGLE PIXEL).

3. Argon_red_spectrum.csv, Argon_blue_spectrum.csv, Neon_red_spectrum.csv, Neon_blue_spectrum.csv andKrypton_red_spectrum.csv give spectrum per physical pixel in 1 second - i.e., the fluxes from the pipeline at a given wavelength have been divided by 3.22. Krypton_blue_spectrum.csv in unavailable at the moment (PIPE2D-941)

Comment by ncaplar [ 11/Nov/21 ]

With the pipeline running on PFI data, I have revisited the question about the difference of flux between predictions and seen results (PIPE2D-902).
The other large difference from results as given in PIPE2D-902 is that I made a mistake when extracting data from tables given by Erin, and extracted the wrong column (amplitude instead of lfux). In short, the end result of this mistakes is that the new results decreases the offset between the prediction and seen results for noble gas lamps by a factor of approximately 2.

In order to deduce the average offset between the predicted and seen flux, I have followed the following procedure:

1. Select 12 brightest lines from the table provided by Erin for the specific lamp and arm
2. Select 12 brightest lines in the data for the specific lamp and arm
3. Find lines which are in both tables 
4. Deduce median multiplicative offset between the predicted and seen fluxes from lines identified in step 3.

For continuum, I compare median fluxes between the predicted and seen fluxes in the area where we have data

Using this procedure, I am finding median offset to be (rounded to the closest integer, which I believe is reasonable given the uncertainties):

Neon, red: 3
Neon, blue: 6
Argon, red: 11
Krypton, red: 6
Continuum (Erin), red: 56
Continuum (Erin), blue: 114

The difference between results in red and blue is probably indicative of the same problem as discussed in PIPE2D-942 and PIPE2D-943.

The figures attached to this ticket show:
Black dots - data predicted by Erin, scaled to be consistent with the flux within a single wavelength element (summed over x direction)
Gray dashed line - wavelength at which Erin predictions are available
Blue line - Original spectrum
Red line - Original spectrum multiplied with the appropriate multiplicative factor, as specified above. 

 

Comment by ncaplar [ 11/Nov/21 ]

I also need to compare the continuum data with a prediction from JEG, that he circulated in an email on November 8, 2021, and that differs from a prediction produced by Erin.

Comment by ncaplar [ 30/Nov/21 ]

I have talked with Erin yesterday (Nov 29). As part of that discussion I promised to close this ticket by Dec 01 by
a) comparing the prediction for continuum from JEG with the data
b) double checking that the fluxes that come from the pipeline (arcLines object) are actually corresponding to the real flux seen in the data.

Comment by ncaplar [ 01/Dec/21 ]

I find that the Jim prediction for continuum that he shared is around 3x times brighter than real data. This is assuming that the flux is linearly proportional to the power supplied (as his prediction is done with 100 W, and 50 W was installed). 3x factor is consistent with Jim has claimed.

Erin has modified her predicted after finding an error on December 2, 2021. After that fix her prediction is around 6x too bright in red, around 11,12 times too bright in blue.

I have double checked, and the results coming from the pipeline seem reasonable.

Erin will write an email to Robert to inquire about further actions and plan to meet with Jim soon.

Generated at Sat Feb 10 15:59:49 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.