The new calibs repo shall be named CALIB-202304-eng.

Here is some feedback on the calib data acquisition from the 2023 April/May engineering run, offered in the hope of improving the observing team's understanding of the requirements, to enable more successful data acquisition in future runs.

• Take quartz exposures as part of the arc sequence (back-to-back, no hexapod movement). The quartz provides a high-quality measurement of the position of the trace in the spatial dimension, so having it improves the quality of the detectorMap.
• Observations with the ring lamp aren't being identified as quartz exposures; I believe this is being addressed.
• Missing simultaneous brn quartz exposures. These are necessary in order to tie the arms together. I suggest that immediately after taking these, we switch to m arm and take some with that too. We should get at least 3 back-to-back repeats.
• Dithered flats were taken in parts with no repeats. We need back-to-back repeat exposures at each dither position for CR removal. Splitting up the sequence into parts is fine, but dither positions should not be repeated (because the positioning isn't perfect, if only because the instrument breathes, so dither=0 now and dither=0 later aren't identical).
• We still lack brm flats, but these are not important.
• The naming scheme for fiber setups could be improved: "1vs4_set1" is not very descriptive. What we need to communicate is the separation between fibers that are active, and the offset. I suggest a scheme something like the following:
  • "modX+Y" means every Xth fiber is illuminated, starting with fiberId=Y.
  • "mod2+1" means every second fiber is illuminated, starting with fiberId=1.
  • "mod8+5" means every fourth fiber is illuminated, starting with fiberId=5.
  • "allHidden" means all fibers are hidden that can be hidden.
  • "allHome" means all fibers are at home that can be driven to home.
• The fiberProfile observations appear to be a strange mix of different observations. The observations I needed are available in the mix (and worked very nicely!), but it looks like many observations were taken that weren't necessary (at least, for the fiberProfile construction), so the time could have been used on something else. What we need for this is something like the following:
  • For fiber configurations allHidden mod4+1 mod4+2 mod4+3 mod4+4 (these can be obtained at different times)
    • For dithers of -0.4, 0.2, 0.0, 0.2, 0.4 pixels:
      • Take a single brn quartz exposure getting about 30k counts at peak in each arm (can be non-simultaneous)
      • Take a single bmn quartz exposure getting about 30k counts at peak in each arm (can be non-simultaneous)
  • For fiber configuration allHome, take multiple simultaneous brn quartz exposures followed by multiple m quartz exposures.
• We lack m and n fiberProfile exposures.

The new b/r 1/3 fiberProfiles (PIPE2D-1220) and a preliminary (and bad) n1 flat were included in CALIB-2023-04-v1, which was released to allow people to start to use them while I improved the n1 flat.
I've added new m1 and m3 fiberProfiles (not the new kind with a large radius done with reduceProfiles due to lack of data, but the old kind with radius=2 done with constructFiberProfiles) and a better n1 flat (PIPE2D-1221) to CALIB-2023-04-v2 and released that.

Thank you for feedback to calibration data in 2023 April/May. This helps us to take good calibration data as well as to estimate overhead in operation.
Could you elaborate why quartz should be taken in sequence of arc data? How much quartz in different time makes a detectorMap worse/difficult to create? In longer-term, sometimes we cannot take all arc data at the same time, so if quartz needs at the same time, it will take more longer.

Also, about "brn" and "bmn" trace (the third bullet), have you seen insufficient repeatability of the red grating movement? Or if you see repeatability of red grating is small enough (in the next run), we can skip moving grating back-to-back? It takes ~2min to move grating, which will count, even if we move it during readout.

The quartz provides a measurement of the location of the trace in x for every row. Arcs may give us at most maybe 100 measurements of the location of the trace in x, so one quartz has in excess of a few tens of times more data than an arc when we consider only the x position. That makes it a very efficient means of constraining the detectorMap.
All arc (and quartz) observations for measuring the detectorMap must be taken back-to-back, with no slit or grating moves or opportunities for the system to change. Because we're measuring the detectorMap, we're not fitting for changes to the detectorMap, and therefore we must assume that the system is identical for all of the contributing exposures. The only way to ensure that with confidence is to take all the observations back-to-back before the system has a chance to change.

I have no data on the repeatability of the red grating movement. My concern in the third bullet point is simply to tie the arms together. A simultaneous brn quartz will allow us to normalise the b,r,n spectra consistently. Then what about m? Well, m doesn't overlap b and n, so it's not terribly important, but it would be good to have a similar normalisation for m. The best way to do that would be to observe m immediately after brn, so the conditions don't change too much.