Are you really running the "full pipeline"? If you're not running parts of the pipeline that require the pfsConfig, you might be able to use the ingestImages.py script instead of ingestPfsImages.py (it may need some config tweaks). Or we could add a flag to ingestPfsImages.py that disables ingestion of the pfsConfigs.

I was trying to run pipeline up to reduce_Arc.py, modeled after pfs_integration_test.sh and the script that I used to reduce data taken in December 2017. I was not aware of the difference between ingestImages.py and ingestPfsImages.py. At the moment this is anyhow somewhat mute for me because I am not able to run full pipeline due to DetectorMap difference discussed in slack and I am cutting my stamps manually. As I discussed with hassan, this a problem that anybody else (e..g, LAM or Japan) who would be trying to follow the manual and/or pfs_integration_test script would encounter. 

One annoyance is that data taken with the LAM sparse slit bundles are precisely when you want/need a pfsConfig, both for the pipeline (constructFiberTrace) and other consumers. (Would it be hard to make a pfsConfig for each of the LAM groups?)

I think we need to take responsibility for providing pfsConfig files for all versions of cable-A. This includes the current LAM dummy cable-A, for which I suspect that Neven's file is fine.

We have a minor problem that there's currently no good place to store these; suggestions welcome — this is the generic boot-strap problem, which also applies to DetectorMap.  Given that we are planning to move to separate products for code and camera properties (i.e. splitting the obs_pfs package), I propose that we invent obs_pfs_data now and put the pfsConfig and pfsDetectorMap files there. Bootstrapping a new repository would put these into the proper place.

One other change that will be needed is to change the actor to write the proper pfsConfigId into the headers; this would be a very good thing anyway.

Can you glance at INFRA-207, which was my bleat for deciding on how to partition data like this before we created a zillion similar repos....

I've pushed a script to drp_stella on branch tickets/PIPE2D-338. ncaplar, could you please check this does what you need?

Anyone who cares about the implementation is welcome to review the GitHub PR.

Grrr, price,unfortunately I am having problem with installing the ticket at the moment, so I am unable to give feedback (but this is completely on my end). Just looking at it, it seems fine. As mentioned, the most crucial part for me is to be able to run reduce_Arc.py that would create detectorMap with correct solutions.

It should be impossible to have "problems installing the ticket" as it's just a git checkout. More to the point, where did "reduce_Arc.py that would create detectorMap with correct solutions" come from? I assume that that's just a typo for reduceArc.py, but the code to update the DetectorMap should be a separate one-off to write an updated DetectorMap followed by registering it. This is separate from reducing arc data.

Ok, sorry for the confusion.
1. I managed to run it (was working in the wrong directory). Running with fibers currently at lam, i.e., makeLamDesign.py blue green red3 red6 creates pfiDesign-0x0000001001000011.fits

2. Indeed, I meant ReduceArc.py. I agree, my comment about DetectorMap was simply wrong. I just wanted to make the point that I wanted to be able to run ReduceArc.py, but that is obvious and did not contribute to the discussion here. I apologize.

I've got ingestPfsImages.py working now so that it if it can't find the pfsConfig file, it will look for the pfiDesign file and use that to create the pfsConfig file, and then ingest it. So I believe this is ready to merge, once people are happy with it.

Just a quick comment : There is no 'red' bundle in the dcb and we can at maximum connect 5 fiber bundles at the same time.

so having fiber [2, 3, 308, 339, 340, 342, 649, 650] together is not possible i'm afraid.

Thanks for letting me know. I've removed the red setting.

Following discussion with @rhl: will proceed with merging branch in order to make progress. Will address subsequent problems that could arise in future tickets.

Merged to master.