Is it necessary to expand the scope of the current sprint to include this? If possible, we should try to finish the tickets we have in play before we move on to anything else.

PIPE2D-80 broke the existing code for tracing the apertures. Before we can merge PIPE2D-80 into master we need to make the FiberTraceSets persistable and change the existing code to include this

Ok, thanks for clarifying.

See obs_pfs and drp_stella tickets/PIPE2D-90.
Quick-start guide has been updated to include constructFiberTrace.
Once this ticket is reviewed and merged with master I will rebase PIPE2D-48, PIPE2D-80, and PIPE2D-77 on master and put them in review as well.

The instructions include the line

      constructFiberTrace.py /Users/azuri/spectra/pfs/PFS --rerun azuri/tmp --id field=FLAT dateObs=2015-12-22 arm=r spectrograph=2 --calibId calibDate=2015-12-22 arm=r spectrograph=2 visit=4

This is inconsistent with the other entries in the file. Do we also need to ingest the fibreTraces?

Also, why do you specify field=FLAT not an explicit visit number, and why is the visit in the calibId?

Is constructFiberTrace pushed to PIPE2D-90?

$ git pull
remote: Counting objects: 1, done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), done.
From github.com:Subaru-PFS/drp_stella
   fdcf7c9..20927fb  master     -> origin/master
Current branch tickets/PIPE2D-90 is up to date.
$ find . -name constructFiberTrace.py
$ 

It's in obs_pfs. (RHL adds: how did I miss that??)

Trying to test the line in the quick-start guide now. 'visit' definitely needs to be replaced with 'calibVersion' since Friday. Fixed the path to $PFS_DATA

Did you push the data model changes? I don't see a tickets/PIPE2D-90 branch, and I don't see a ticket for this change to the data model.

You didn't reply to my question about ingestion; it seems inconsistent with the other calibration products like flats not to need to ingest the fiberTraces.

Fixed the command in the quick-start guide and tested it. You're right - pfsFiberTraces do need to get ingested into the CALIB database. I replaced field=Flat with visit=5 to be consistent with the previous line reduceFlat...
Sorry, the datamodel changes are in PIPE2D-68. Should have mentioned that...

Hmm. Don't I need to specify calibVersion=fiberTrace in the --calibId? And if I add that I get
{{
File "/Users/rhl/LSST/obs/pfs/python/lsst/obs/pfs/pfsMapper.py", line 97, in map
location = super(PfsMapper, self).map(datasetType, copyId, write=write)
File "/Users/lsst/products/DarwinX86/miniconda/latest/envs/lsst-v12_1/opt/lsst/daf_persistence/python/lsst/daf/persistence/mapper.py", line 172, in map
func = getattr(self, 'map_' + datasetType)
AttributeError: 'PfsMapper' object has no attribute 'map_linearizer'
}}

That's
{{
$ cd /Users/rhl/LSST/obs/pfs
$ git pull
Current branch tickets/PIPE2D-90 is up to date.
}}
so I think I have the right mapper.

Can you modify the runDemo script that I sent you and confirm that it works?

Also, I'm surprised to see that it uses `parseAndRun()` not `parseAndSubmit()` like the other tasks (e.g. reduceBias.py). Can you explain?

Please pull drp_stella and obs_pfs again (just did some changes). The 'map_linearizer' error normally shows up when the 'config.isr.doLinearize = False' is commented out what I only see in obs_pfs/config/pfs/dark.py.
I replaced 'parseAndRun()' with 'parseAndSubmit()' so now the parameter '--cores 1' needed to get added to the command line for constructFiberTrace... (change pushed in drp_stella)

I was unable to confirm that the code is correct (as the full processing relies on PIPE2D-48) but it does run! I'm passing this review for now, but we need to confirm that it's doing the right thing as part of the PIPE2D-48 review.

Created issue to write test for pfsFiberTrace (PIPE2D-98). Merged into master