We discussed this at the meeting of 2017-04-14. Working on this is the primary goal for the next week; we expect aritter to report on progress at the meeting of 2017-04-21.

please also provide documentation, examples, and file more tickets as needed (e.g. the calibration did not fail hard enough,...)

The method for the line list creation is as follows:

• combine the NIST line lists for the individual elements of the Arc lamps
• extract the Arc FiberTrace
• create an artificial spectrum from the NIST wavelengths and predicted strengths
• cross-correlate the Arc spectrum with the artificial spectrum to find the offset
• fit each line with a Gaussian to find the position and strength
• remove lines which have a measured strength of less than minStrength, blends, and lines which could not be fit or are flagged as problematic

Blends are identified as follows:
For each line search for lines with a fitted position close to its own position (within minDist * fwhm pixels)
If a possible blend is identified, compare the predicted lines strengths.
If one line is at least minRatio times stronger than the other lines in the blend, keep the strong line, otherwise discard all lines in the blend

Split into different tickets where it makes sense:

• Please add diagnostic tools, e.g. take a pfsArm file and overplot the lines from the line lists (full and good lines only)
• keep as much information in memory as possible and return it so we can persist it later if we want to
• create an example jypiter notebook to run the task and create diagnostic plots
• if you take 2 Arcs and run the full wavelength calibration for one and apply it to the 2nd, how good is the RMS?
• plot the dispersions of adjacent/all wavelength solutions
• safe enough information to recalculate dispRMS and dispRMSCheck
• make sure that all the data read, used, mentioned, written are according to the datamodel (nm or Ang, vac or air)

write utility functions that read back pfsArm files and plot useful information
safe offset from cross correlation
file DAMD PR to add 2 HDUs which will contain all the line information - 1 for NIST input, 1 for measurements

To compile/run you will need the branches tickets/PIPE2D-139 for obs_pfs and tickets/DAMD-17 for datamodel.
The example ipython notebook is in drp_stella/examples/reduceArc.ipynb

The example jupyter notebook which demonstrates how to create the master line list is examples/line lists.ipynb

@aritter What does your constructArc.py script do? Your example passes in visit 103 and then uses 103 as the argument to reduceArc – is this a coincidence? In other words, does one run constructArc.py once and then have the ability to reduce any combination of arcs?

I'm confused. There is no constructArc.py in examples/line lists.ipynb. constructArc.py did the ISR on the arc is - I believe - obsolete since you changed reduceArcTask.py to take the detrend}}ed {{postISR file

Yes, you're right. Trying to do too many things.

This had too many unrelated changes.

We'll redo this by reading input (NIST) files for different elements; this seems to work