On the sims, I get:

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.154053 nm (1.816 pix) from 507/507 (0.079249 nm = 0.934 pix for 4 reserved points)

Note the large number of lines. This is because it doesn't know what kind of arc it is:

reduceArc INFO: Arc lamp elements are: 

If I remove all the reference lines except for Ne, I get:

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.010457 nm (0.123 pix) from 58/58 (0.002898 nm = 0.034 pix for 4 reserved points)

which is reasonable. We need to make sure that we can discover the kind of arc from the sims data (the same way it's set for the real data).

I suspect the problem with the LAM HgAr data I was looking at is that the line list hasn't been vetted, so blends are making line identification difficult.

The main problem with the HgAr arcs was from centroiding on noise. The algorithm we have been using for identifying lines (implemented primarily in Spectrum::identify) attempts to centroid on each of the reference lines. However, most of the Hg reference lines aren't detected in our spectra (they're mostly Ar with a few bright Hg lines; I did have to restore some which had been overzealously flagged a time or two I looked at wavelength solutions before), which means the centroids were noisy. I added a new line identification algorithm that uses the FindLinesTask (introduced in PIPE2D-431) and then matches against the reference lines (much as we do when matching stars for astrometry in LSST/HSC). This produced wavelength solutions that were better but not good enough (several tenths of a pixel); but the quality of the solutions were highly variable from fiber to fiber. On further investigation, I rediscovered the fact that the HgAr arcs contain continuum, which the FindLinesTask was not accounting for in the threshold, which meant that, again, we were centroiding on noise. With all that fixed, I get for HgAr visit=5825 (with no vetting of the line list):

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.008128 nm (0.096 pix) from 25/28 (0.004550 nm = 0.054 pix for 4 reserved points)

This sounds good, but I don't understand why you had to add a new line ID task. With a trimmed line list to only the good ones and a DetectorMap good to a pixel or so from bootstrapping on Ne I'd have expected to only try to centroid bright lines; I take your continuum point, but I'm surprised that that matters for bright lines.
If we're using a low-order Chebyshev and a decent DetectorMap the residuals should be pretty small.

This is not to say that your algorithm to use more fainter lines isn't good, I just don't understand why we need it to get decent HgAr solutions

I guess I didn't think about flagging lines just because they're faint. Wouldn't we want to use the same line list with a long exposures as with short exposures?

I believe much of the problem was due to the detectorMap, but there are a lot of little fixes on the ticket branch that also proved useful not only in fixing little problems, but also diagnosing problems along the way. I found that a separate detectorMap was needed for the 2017 September data (no surprise, but we hadn't gone to the trouble of doing that), and uploaded it to the DRP wiki.

At this point, with separate bootstrapped detectorMaps for the 2017 September and 2019 April data, I get:

2017 September Ne (visit=5830):

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.004793 nm (0.056 pix) from 49/58 (0.005559 nm = 0.065 pix for 4 reserved points), 650.83-955.00 nm

2017 September HgAr (visit=5825):

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.003706 nm (0.044 pix) from 23/27 (0.201716 nm = 2.376 pix for 4 reserved points), 671.82-966.04 nm

(The large reserved RMS is because the highest-wavelength line is being selected as being reserved, which has a large effect on the fit; I expect this will be addressed when we do wavelength calibration with multiple arc flavors.)

2019 April Ne (visit=16292):

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.010284 nm (0.121 pix) from 26/29 (0.012353 nm = 0.145 pix for 4 reserved points), 640.40-942.80 nm

2019 April HgAr (visit=17071):

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.005257 nm (0.062 pix) from 27/32 (0.006229 nm = 0.073 pix for 4 reserved points), 671.82-966.04 nm

2019 April Kr (visit=17809):

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.009664 nm (0.114 pix) from 30/33 (0.006577 nm = 0.077 pix for 4 reserved points), 690.61-936.47 nm

I think this is ready to review. There are changes to obs_pfs, drp_stella, drp_instmodel, drp_instdata and pfs_pipe2d.

Changes acceptable.

Merged to master and tagged as 5.0.5.