So what do we do about this PITA? As a reminder, pfs_utils is the package dedicated to code shared between the ICS and DRP worlds (fiber id mappings, core coordinate transforms, etc), and was supposed to not require any other ICS or DRP packages.

But for DRP (scons) installation and CI testing, sconsUtils has been added, which drags in a bunch of DRP infrastructure. Instead of being a trivial to install standalone package, it forces everyone to install some part of the Rubin universe. Subaru ICS, MPA, ASRD are the

So pull sconsUtils out? Add it and its friends as a requirement? The LSST requirement on some utterly antique pytest and thus attrs has already given us (ICS) trouble; with my ICS hat on I am very disinclined to buy into any more dependencies.

If it helps, this is not really urgent on my side. I currently just copy the Python files to a place where the interpreter finds them.

Perhaps it would already be sufficient to add a very basic setup.py.

Agreed on adding basic setup.py.

I'm encountering the same problem with "datamodel". It would be really great if this issue could be addressed for all concerned packages.

In the meantime I'll have to copy Python files around by hand...

Martin Reinecke: Can you provide the most recent steps you made, and a copy of the error message you saw please?

It's not so much a matter of commands failing, but rather of no well-defined installation procedure existing.

As I mentioned in the first post, I'm not aware of any document describing how these packages should be installed. I tried finding out by myself, but it requires manual installation of a not very common build tool (scons), then additionally cloning lsst.sconsUtils (also manually). Last time I tried, the package didn't install even then, but that's not really the essence of this issue.

The proposed solution (add a simple setup.py script) seems to have general agreement, but it's not implemented yet.

I will back off the setup.py suggestion and push for using requiring basic eups instead. Basically, all the PFS packages (both ICS and DRP) use a package directory tree which does not work well with the python setuptool/distutils/pkg_resources policies. Specifically, we expect a top-level directory with bin/, etc/, docs/, python/pfs/utils/, data/, examples/, etc. in it, and use our own mechanism to adjust PYTHONPATH and PATH to match. The builtin tools put the content of our python directory into ...../site-packages/, merging all pfs packages into that, and leave PYTHONPATH alone. Extra files and directories are put in other, non-site-packages directories.

This matters for pfs_utils, which includes some stable but essential data files in data/fiberids/, and which are found by using eups routines.

One could certainly argue that what we do is not what one would start with in 2020, but we started much earlier and this is what we have. We can semi-cheat and get something close to what PFS does, but not without depending on undocumented behavior. Using bare-bones eups ("setup -r .") seems like a not-insane middle ground.

OK, so eups. I think there are some basic intro docs both in the Rubin/HSC and the PFS worlds. Someone might have a decent pointer....

I have added a skeleton `setup.py` file on the tickets/INSTRM-1047 branch of the datamodel repo, which does the trick for me. I also see that cloomis has been working on this for pfs_utils, but I don't manage to get his solution to work for me yet.

If wanted, I can provide a simple setup.py script for pfs_utils as well. Please let me know!

I took the liberty of pushing a setup.py file to the tickets/INSTRM-1047 branch of the pfs_utils repository. Please feel free to remove it again if a different sort of solution is desired!

What still needs to happen to close this ticket and merge the changes?

I merged datamodel and pfs_utils.

Thank you!

Can we close this ticket?

I'd say yes. If anything else turns up, we can open another one.

Do I have to do special Jira magic so that it can be closed?