[PIPE2D-483] Process data for DRP2D report Created: 05/Dec/19  Updated: 19/Feb/20  Resolved: 19/Feb/20

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: Normal
Reporter: price Assignee: price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 4
Sprint: 2DDRP-2019 J, 2DDRP-2019 K
Reviewers: hassan

 Description   

rhl writes:

I'd like to discuss

  • fibre extraction
  • Wavelength calibration
  • PSF estimation
  • b/r/[n] combination

and requests the following:

For fibre extraction, I think we'll neeed figures of

  • fibre flats
  • fibre flats overlaid with the fibre traces
  • residuals of the fibre flats with the (extracted spectra x fibre traces) removed

Then for the arcs

  • Raw arcs (Ne, HgAr, Xe)
  • Extracted raw arc spectra (i.e. in pixels)
  • Extracted lambda-calibrated arc spectra
  • Figures showing the wavelength solution plots (and tables of RMSs)
  • Evolution of wavelength solutions with time
  • Probably need a figure of the DetectorMaps
  • Something about continuum estimation, probably a before-and-after 2-D image

For PSF estimation:
A reminder of Neven's work, probably a list of all the components going into the model
(with pictures as appropriate) followed by

  • arcs
  • arcs with PSFs subtracted
  • extracted arcs lines
  • extracted residuals (with error bars)

For b/r/[n] extraction I think we need to show

  • b1 arcs
  • r1 arcs
  • combined arcs

 Comments   
Comment by price [ 06/Dec/19 ]

I'm working in /tigress/pprice/pipe2d-483.

Here's some code to produce some fiber extraction plots:

from lsst.daf.persistence import Butler
from lsst.afw.display import Display
import numpy as np
backend = "ds9"

butler = Butler("/tigress/pprice/pipe2d-483/DATA/rerun/calibs/fiberTrace")
dataId = dict(visit=21122, arm="r", spectrograph=1)
quartz = butler.get("postISRCCD", dataId)
detMap = butler.get("detectormap", dataId)
traces = butler.get("fibertrace", dataId)

# Plot traces on top of quartz
display = Display(1, backend)
display.mtv(quartz)
step = 100  # Looks like display_ds9 can't handle a really long list of points for the line
with display.Buffering():
    for tt in traces:
        display.line(list(zip(detMap.getXCenter(tt.fiberId)[::step], np.arange(tt.trace.getHeight(), step=step))))

# Show fiberTrace residuals
spectra = traces.extractSpectra(quartz.maskedImage)
for tt, ss in zip(traces, spectra):
    image = tt.constructImage(ss)
    quartz.maskedImage[image.getBBox()] -= image
Display(2, backend).mtv(quartz)
Comment by price [ 07/Dec/19 ]

For the arcs, I've run the following under /tigress/pprice/pipe2d-483/DATA with --rerun=arc:

  • visit=21460: Ne
  • visit=21664: Kr
  • visit=21868: Xe
  • visit=21256: HgAr

Note that the Kr and Xe linelists are completely untuned (I haven't run any of those before), and blue Ne is only mildly tuned; red Ne has had a decent amount of attention.

There's a continuum-subtracted rerun for the HgAr: arcContinuumSub. It didn't work on the blue arm (not yet sure why).

You should be able to get the calExp images, and pfsArm files from the above rerun.

I've put some of the wavelength calibration debug plots under /tigress/pprice/pipe2d-483:

bootstrap_fit.png           Ne_blue_463.png        Ne_red_339.png
bootstrap_shifts.png        Ne_blue_524.png        Ne_red_339_wavelength.png
Ne_blue_192.png             Ne_blue_586.png        Ne_red_401.png
Ne_blue_255.png             Ne_blue_63.png         Ne_red_464.png
Ne_blue_2.png               Ne_blue_649.png        Ne_red_525.png
Ne_blue_338_pixels.png      Ne_red_192.png         Ne_red_587.png
Ne_blue_338.png             Ne_red_255.png         Ne_red_63.png
Ne_blue_338_wavelength.png  Ne_red_2.png           Ne_red_650.png
Ne_blue_400.png             Ne_red_339_pixels.png

Most of these are plots of the wavelength solution. The *_pixels.png and *_wavelength.png plots are spectra from a middle fiber, with lines marked.

The bootstrap plots are made using 2019 July data, relative to a 2019 April detectorMap; the command was:

bootstrapDetectorMap.py DATA --calib DATA/CALIB --rerun bootstrap --flatId visit=21122 --arcId visit=21460 arm=r

Note that there's a fiberId difference between the red and the blue (the code doesn't look at the pfsConfig to figure out what's illuminated).

Here are the logs for each:

Ne blue:

reduceArc.calibrateWavelengths INFO: FiberId 649, rms 0.006184 nm (0.091 pix) from 26/30 (0.005071 nm = 0.074 pix for 4 reserved points), 540.21-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 649: wavelength correction -127.470261 +/- 251.732590 nm
reduceArc.calibrateWavelengths INFO: FiberId 586, rms 0.001703 nm (0.025 pix) from 25/29 (0.026191 nm = 0.384 pix for 4 reserved points), 534.26-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 586: wavelength correction -39.869957 +/- 78.290649 nm
reduceArc.calibrateWavelengths INFO: FiberId 524, rms 0.006478 nm (0.095 pix) from 28/30 (0.013654 nm = 0.200 pix for 4 reserved points), 534.26-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 524: wavelength correction 5.990717 +/- 11.937734 nm
reduceArc.calibrateWavelengths INFO: FiberId 463, rms 0.004519 nm (0.066 pix) from 29/32 (0.005444 nm = 0.080 pix for 4 reserved points), 448.94-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 463: wavelength correction -2.652970 +/- 6.735330 nm
reduceArc.calibrateWavelengths INFO: FiberId 400, rms 0.007545 nm (0.111 pix) from 30/31 (0.136676 nm = 2.005 pix for 4 reserved points), 511.79-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 400: wavelength correction -2.776120 +/- 8.448882 nm
reduceArc.calibrateWavelengths INFO: FiberId 338, rms 0.005525 nm (0.081 pix) from 26/30 (0.006392 nm = 0.094 pix for 4 reserved points), 540.21-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 338: wavelength correction -43.377213 +/- 88.127434 nm
reduceArc.calibrateWavelengths INFO: FiberId 255, rms 0.146800 nm (2.154 pix) from 27/29 (0.169655 nm = 2.489 pix for 4 reserved points), 533.72-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 255: wavelength correction 913.316833 +/- 1834.392456 nm
reduceArc.calibrateWavelengths INFO: FiberId 192, rms 0.146342 nm (2.147 pix) from 27/29 (0.171984 nm = 2.523 pix for 4 reserved points), 533.72-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 192: wavelength correction 858.033081 +/- 1722.531372 nm
reduceArc.calibrateWavelengths INFO: FiberId 63, rms 0.019850 nm (0.291 pix) from 26/28 (0.026201 nm = 0.384 pix for 4 reserved points), 534.26-653.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 63: wavelength correction 17.563498 +/- 32.926407 nm
reduceArc.calibrateWavelengths INFO: FiberId 2, rms 0.003770 nm (0.055 pix) from 24/26 (0.002229 nm = 0.033 pix for 4 reserved points), 534.26-650.83 nm
reduceArc.calibrateWavelengths INFO: Fiber 2: wavelength correction 89.346375 +/- 177.462524 nm

Ne red:

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.014837 nm (0.175 pix) from 53/60 (0.039319 nm = 0.463 pix for 4 reserved points), 640.40-948.93 nm
reduceArc.calibrateWavelengths INFO: Fiber 650: wavelength correction 0.081345 +/- 0.045970 nm
reduceArc.calibrateWavelengths INFO: FiberId 587, rms 0.018194 nm (0.213 pix) from 54/60 (0.028102 nm = 0.329 pix for 4 reserved points), 638.48-953.68 nm
reduceArc.calibrateWavelengths INFO: Fiber 587: wavelength correction 0.031532 +/- 0.038683 nm
reduceArc.calibrateWavelengths INFO: FiberId 525, rms 0.008649 nm (0.101 pix) from 50/60 (0.001389 nm = 0.016 pix for 4 reserved points), 640.40-955.00 nm
reduceArc.calibrateWavelengths INFO: Fiber 525: wavelength correction 0.007091 +/- 0.040119 nm
reduceArc.calibrateWavelengths INFO: FiberId 464, rms 0.007401 nm (0.086 pix) from 49/59 (0.005413 nm = 0.063 pix for 4 reserved points), 640.40-955.00 nm
reduceArc.calibrateWavelengths INFO: Fiber 464: wavelength correction -0.011258 +/- 0.033281 nm
reduceArc.calibrateWavelengths INFO: FiberId 401, rms 0.022182 nm (0.257 pix) from 55/60 (0.006854 nm = 0.080 pix for 4 reserved points), 640.40-953.68 nm
reduceArc.calibrateWavelengths INFO: Fiber 401: wavelength correction 0.013599 +/- 0.026009 nm
reduceArc.calibrateWavelengths INFO: FiberId 339, rms 0.019939 nm (0.231 pix) from 55/59 (0.010686 nm = 0.124 pix for 4 reserved points), 638.48-948.93 nm
reduceArc.calibrateWavelengths INFO: Fiber 339: wavelength correction 0.022436 +/- 0.047537 nm
reduceArc.calibrateWavelengths INFO: FiberId 255, rms 0.007695 nm (0.089 pix) from 52/60 (0.011089 nm = 0.129 pix for 4 reserved points), 640.40-955.00 nm
reduceArc.calibrateWavelengths INFO: Fiber 255: wavelength correction -0.047651 +/- 0.016393 nm
reduceArc.calibrateWavelengths INFO: FiberId 192, rms 0.009561 nm (0.111 pix) from 48/60 (0.007089 nm = 0.083 pix for 4 reserved points), 650.83-955.00 nm
reduceArc.calibrateWavelengths INFO: Fiber 192: wavelength correction -0.049444 +/- 0.007154 nm
reduceArc.calibrateWavelengths INFO: FiberId 63, rms 0.012329 nm (0.145 pix) from 52/59 (0.012291 nm = 0.144 pix for 4 reserved points), 640.40-955.00 nm
reduceArc.calibrateWavelengths INFO: Fiber 63: wavelength correction 0.036073 +/- 0.020439 nm
reduceArc.calibrateWavelengths INFO: FiberId 2, rms 0.010217 nm (0.120 pix) from 50/58 (0.009391 nm = 0.111 pix for 4 reserved points), 640.40-955.00 nm
reduceArc.calibrateWavelengths INFO: Fiber 2: wavelength correction 0.106759 +/- 0.029395 nm

Kr blue: Not anywhere near enough lines!

Kr red:

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.002652 nm (0.031 pix) from 22/27 (0.006560 nm = 0.077 pix for 4 reserved points), 645.81-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 650: wavelength correction 0.095861 +/- 0.065640 nm
reduceArc.calibrateWavelengths INFO: FiberId 587, rms 0.017134 nm (0.201 pix) from 28/29 (0.013514 nm = 0.158 pix for 4 reserved points), 690.66-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 587: wavelength correction 0.056022 +/- 0.046040 nm
reduceArc.calibrateWavelengths INFO: FiberId 525, rms 0.003142 nm (0.037 pix) from 23/31 (0.135587 nm = 1.583 pix for 4 reserved points), 645.81-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 525: wavelength correction 0.076505 +/- 0.158832 nm
reduceArc.calibrateWavelengths INFO: FiberId 464, rms 0.012909 nm (0.150 pix) from 31/32 (0.222272 nm = 2.586 pix for 4 reserved points), 645.81-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 464: wavelength correction 0.066283 +/- 0.214989 nm
reduceArc.calibrateWavelengths INFO: FiberId 401, rms 0.013168 nm (0.153 pix) from 30/32 (0.018864 nm = 0.219 pix for 4 reserved points), 645.81-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 401: wavelength correction 0.034724 +/- 0.085044 nm
reduceArc.calibrateWavelengths INFO: FiberId 339, rms 0.011314 nm (0.131 pix) from 26/29 (0.016115 nm = 0.187 pix for 4 reserved points), 690.66-898.04 nm
reduceArc.calibrateWavelengths INFO: Fiber 339: wavelength correction 0.869967 +/- 1.735318 nm
reduceArc.calibrateWavelengths INFO: FiberId 255, rms 0.004866 nm (0.057 pix) from 24/27 (0.004350 nm = 0.051 pix for 4 reserved points), 722.61-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 255: wavelength correction -0.809603 +/- 1.909085 nm
reduceArc.calibrateWavelengths INFO: FiberId 192, rms 0.004894 nm (0.057 pix) from 24/27 (0.002350 nm = 0.027 pix for 4 reserved points), 690.61-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 192: wavelength correction -0.006117 +/- 0.092426 nm
reduceArc.calibrateWavelengths INFO: FiberId 63, rms 0.015359 nm (0.180 pix) from 22/23 (0.017343 nm = 0.203 pix for 4 reserved points), 722.61-936.47 nm
reduceArc.calibrateWavelengths INFO: Fiber 63: wavelength correction 1.238212 +/- 2.972113 nm
reduceArc.calibrateWavelengths INFO: FiberId 2, rms 0.078861 nm (0.929 pix) from 24/25 (0.031607 nm = 0.372 pix for 4 reserved points), 645.81-893.11 nm
reduceArc.calibrateWavelengths INFO: Fiber 2: wavelength correction 0.880845 +/- 2.127147 nm

Xe blue:

reduceArc.calibrateWavelengths INFO: FiberId 649, rms 0.027418 nm (0.403 pix) from 19/22 (0.030593 nm = 0.449 pix for 4 reserved points), 458.27-631.81 nm
reduceArc.calibrateWavelengths INFO: Fiber 649: wavelength correction 5.211668 +/- 13.130107 nm
reduceArc.calibrateWavelengths INFO: FiberId 586, rms 0.004167 nm (0.061 pix) from 16/19 (0.072546 nm = 1.064 pix for 4 reserved points), 467.12-631.81 nm
reduceArc.calibrateWavelengths INFO: Fiber 586: wavelength correction -0.696402 +/- 2.053455 nm
reduceArc.calibrateWavelengths INFO: FiberId 524, rms 0.009736 nm (0.143 pix) from 16/21 (0.013517 nm = 0.198 pix for 4 reserved points), 458.27-631.81 nm
reduceArc.calibrateWavelengths INFO: Fiber 524: wavelength correction 0.442230 +/- 0.855534 nm
reduceArc.calibrateWavelengths INFO: FiberId 463, rms 0.009626 nm (0.141 pix) from 20/25 (2.292654 nm = 33.632 pix for 4 reserved points), 450.10-647.28 nm
reduceArc.calibrateWavelengths INFO: Fiber 463: wavelength correction -1.423853 +/- 3.113337 nm
reduceArc.calibrateWavelengths INFO: FiberId 400, rms 0.009408 nm (0.138 pix) from 14/21 (0.140962 nm = 2.068 pix for 4 reserved points), 467.12-647.28 nm
reduceArc.calibrateWavelengths INFO: Fiber 400: wavelength correction -26.732361 +/- 66.545532 nm
reduceArc.calibrateWavelengths INFO: FiberId 338, rms 0.026269 nm (0.385 pix) from 15/19 (0.033615 nm = 0.493 pix for 4 reserved points), 462.43-647.28 nm
reduceArc.calibrateWavelengths INFO: Fiber 338: wavelength correction -5.307633 +/- 14.372238 nm
reduceArc.calibrateWavelengths INFO: FiberId 255, rms 0.099594 nm (1.461 pix) from 18/19 (0.157716 nm = 2.314 pix for 4 reserved points), 467.12-646.97 nm
reduceArc.calibrateWavelengths INFO: Fiber 255: wavelength correction 16.561207 +/- 41.482056 nm
reduceArc.calibrateWavelengths INFO: FiberId 192, rms 0.009100 nm (0.133 pix) from 18/23 (0.026921 nm = 0.395 pix for 4 reserved points), 452.47-646.97 nm
reduceArc.calibrateWavelengths INFO: Fiber 192: wavelength correction -0.478070 +/- 0.821675 nm
reduceArc.calibrateWavelengths INFO: FiberId 63, rms 0.000000 nm (0.000 pix) from 11/13 (0.565573 nm = 8.297 pix for 4 reserved points), 467.12-631.81 nm
reduceArc.calibrateWavelengths INFO: Fiber 63: wavelength correction 868.787048 +/- 2131.654541 nm
reduceArc.calibrateWavelengths INFO: FiberId 2, rms inf nm (inf pix) from 10/12 (0.010947 nm = 0.161 pix for 4 reserved points), 467.12-631.81 nm
reduceArc.calibrateWavelengths INFO: Fiber 2: wavelength correction -0.772644 +/- 5.210824 nm

Xe red:

reduceArc.calibrateWavelengths INFO: FiberId 650, rms 0.101127 nm (1.190 pix) from 104/114 (0.036807 nm = 0.433 pix for 4 reserved points), 632.58-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 650: wavelength correction 0.247238 +/- 0.117824 nm
reduceArc.calibrateWavelengths INFO: FiberId 587, rms 0.028141 nm (0.330 pix) from 95/116 (0.022299 nm = 0.261 pix for 4 reserved points), 646.97-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 587: wavelength correction 0.249946 +/- 0.074510 nm
reduceArc.calibrateWavelengths INFO: FiberId 525, rms 0.016808 nm (0.196 pix) from 91/118 (0.011665 nm = 0.136 pix for 4 reserved points), 632.58-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 525: wavelength correction 0.220641 +/- 0.066134 nm
reduceArc.calibrateWavelengths INFO: FiberId 464, rms 0.025147 nm (0.293 pix) from 96/118 (0.009348 nm = 0.109 pix for 4 reserved points), 646.97-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 464: wavelength correction 0.203063 +/- 0.062553 nm
reduceArc.calibrateWavelengths INFO: FiberId 401, rms 0.024540 nm (0.285 pix) from 95/116 (0.006920 nm = 0.080 pix for 4 reserved points), 648.78-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 401: wavelength correction 0.232160 +/- 0.059687 nm
reduceArc.calibrateWavelengths INFO: FiberId 339, rms 0.074964 nm (0.869 pix) from 104/115 (0.038755 nm = 0.449 pix for 4 reserved points), 632.58-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 339: wavelength correction 0.223555 +/- 0.090384 nm
reduceArc.calibrateWavelengths INFO: FiberId 255, rms 0.020384 nm (0.237 pix) from 96/115 (0.036662 nm = 0.426 pix for 4 reserved points), 646.97-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 255: wavelength correction 0.180039 +/- 0.061076 nm
reduceArc.calibrateWavelengths INFO: FiberId 192, rms 0.015441 nm (0.180 pix) from 89/111 (0.008399 nm = 0.098 pix for 4 reserved points), 646.97-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 192: wavelength correction 0.176159 +/- 0.027350 nm
reduceArc.calibrateWavelengths INFO: FiberId 63, rms 0.077166 nm (0.905 pix) from 93/106 (0.048656 nm = 0.571 pix for 4 reserved points), 647.28-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 63: wavelength correction 0.176646 +/- 0.143441 nm
reduceArc.calibrateWavelengths INFO: FiberId 2, rms 0.135710 nm (1.598 pix) from 94/101 (0.053325 nm = 0.628 pix for 4 reserved points), 647.28-970.10 nm
reduceArc.calibrateWavelengths INFO: Fiber 2: wavelength correction 0.234547 +/- 0.084517 nm

If you want to run an arc yourself, the command is like:

reduceArc.py /tigress/pprice/pipe2d-483/DATA --calib /tigress/pprice/pipe2d-483/DATA/CALIB --rerun arc --id visit=21460 arm=b
Comment by rhl [ 08/Dec/19 ]

Can you rerun those calibrations with order=2 (or maybe 3 – whatever works). Please include the visit number in the text giving the RMS (or did Keigo's work to dump these numbers get finished? If so, can you remind me how to access the data and make the plots). And please remake the debug plots, again including the visit numbers, with the new orders. Could we have some more elements too (for selected fibres?).

Also, can we look at how well the derived wavelength solutions (i.e. the updated DetectorMaps) work with independent arcs – I think that the debug plots should manage that with order=0, although that'd be a hack.

Looking at e.g. Ne_blue_400.png my guess is that there weren't any Ne lines in the Blue, but it'd be nice to have one of the _pixels or _wavelength plots to be sure (which ever is clearer).

I didn't see any merged arc spectra yet. Does that code work? If not, can you generate some figures from the sims.

Comment by price [ 10/Dec/19 ]

There are new plots available at /tigress/pprice/pipe2d-483:

  • Ne_21460b1.png
  • Ne_21460r1.png
  • Xe_21868b1.png
  • Xe_21868r1.png
  • Kr_21664b1.png
  • Kr_21664r1.png

The stats for the middle fiber:

Ne 21460b1:

reduceArc.calibrateWavelengths INFO: FiberId 338, rms 0.004961 nm (0.073 pix) from 27/31 (0.018217 nm = 0.268 pix for 10 reserved points), 534.48-653.47 nm

Ne 21460r1:

reduceArc.calibrateWavelengths INFO: FiberId 339, rms 0.009856 nm (0.114 pix) from 52/59 (0.012415 nm = 0.144 pix for 10 reserved points), 640.40-955.00 nm

Kr 21664b1:

reduceArc.calibrateWavelengths INFO: FiberId 338, rms inf nm (inf pix) from 3/3 (nan nm = nan pix for 0 reserved points), 556.38-587.25 nm

Kr 21664r1:

reduceArc.calibrateWavelengths INFO: FiberId 339, rms 0.005385 nm (0.062 pix) from 23/29 (0.003935 nm = 0.046 pix for 10 reserved points), 690.66-936.47 nm

Xe 21868b1:

reduceArc.calibrateWavelengths INFO: FiberId 338, rms 0.174265 nm (2.563 pix) from 19/19 (0.215072 nm = 3.163 pix for 10 reserved points), 462.43-647.28 nm

Xe 21868r1:

reduceArc.calibrateWavelengths INFO: FiberId 339, rms 0.089789 nm (1.042 pix) from 101/114 (0.087491 nm = 1.015 pix for 10 reserved points), 632.58-970.10 nm

Comment by price [ 10/Dec/19 ]

For PSFs and 2D sky subtraction, see the reruns sky2d and noSky2d. There are calExp and pfsArm datasets to look at.

I ran (with tickets/PIPE2D-326 of drp_stella):

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483 $ reduceArc.py DATA --calib DATA/CALIB --rerun sky2d --id visit=21460
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483 $ cp DATA/rerun/sky2d/DETECTORMAP/pfsDetectorMap-021460-r1.fits DATA/CALIB/DETECTORMAP/pfsDetectorMap-000000-r1.fits 
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483 $ reduceExposure.py DATA --calib DATA/CALIB --rerun sky2d  --id visit=21460 arm=r -c doMeasurePsf=True doSubtractSky2d=True fiberDx=-0.13 doWriteCalexp=True subtractSky2d.useAllFibers=True

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483 $ reduceExposure.py DATA --calib DATA/CALIB --rerun noSky2d  --id visit=21460 arm=r -c fiberDx=-0.13 doWriteCalexp=True
Comment by price [ 10/Dec/19 ]

Oh, for the arcs that I ran earlier, see the rerun arc.

I ran:

reduceArc.py DATA --calib DATA/CALIB --rerun arc --id visit=21460 arm=b
reduceArc.py DATA --calib DATA/CALIB --rerun arc --id visit=21460 arm=r
reduceArc.py DATA --calib DATA/CALIB --rerun arc --id visit=21664 arm=b
reduceArc.py DATA --calib DATA/CALIB --rerun arc --id visit=21664 arm=r
reduceArc.py DATA --calib DATA/CALIB --rerun arc --id visit=21868 arm=b
reduceArc.py DATA --calib DATA/CALIB --rerun arc --id visit=21868 arm=r
Comment by price [ 10/Dec/19 ]

For extraction demonstrations, I used the integration test (to work around the b/r fiberId mismatches in the LAM data), in /tigress/pprice/pipe2d-483/integrationTest/INTEGRATION. There's two things to look at there:

  • rerun=integration/pipeline has simulated data run all the way through the pipeline. I did this with PIPE2D-391, so there's 600 fibers, and I expect you can find a nice simulated galaxy to show off.
  • rerun=extraction has an arc that has been run through reduceExposure and mergeArms, so you can look at the pfsArm and pfsMerged files.

I ran:

(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483 $ mkdir integrationTest
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483/integrationTest pfs_integration_test.sh -c 8 -n -b tickets/PIPE2D-391 .
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483/integrationTest $ reduceExposure.py INTEGRATION --calib INTEGRATION/CALIB --rerun extraction --id field=ARC -c doSubtractSky2d=False
(lsst-scipipe) pprice@tiger2-sumire:/tigress/pprice/pipe2d-483/integrationTest $ mergeArms.py INTEGRATION --calib INTEGRATION/CALIB --rerun extraction --id field=ARC
Comment by price [ 13/Dec/19 ]

The talk is done.

Comment by price [ 15/Feb/20 ]

This work generated some commits on drp_stella that I'd like to merge to master.

Comment by price [ 19/Feb/20 ]

Merged to master.

Generated at Sat Feb 10 15:53:56 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.