I think that the necessary information to identify proposals and objects for tracking progress is ob_code (string) and proposal_id (string) (or group_id (string)) from targetdb. Which is easier, to add columns to the existing extension (perhaps 1st extension) or to add a new extension including such operation-related columns in pfsDesign and pfsConfig?

I suggest we put it in the existing main extension with everything else, and give it a default value if it's not present (old files).

So, you'd add a proposal_id attribute with a value per fiber, so 2458 values, is that right ?

I believe we need to associate a proposal with each fiber. I think the options are to have a proposalId for each fiber (in the same way each fiber has an ra, dec, etc), or we could write a new extension with proposalId associated with a variable-length array of fiberId. I like the former, because it naturally forces a proposal to be associated with each fiber, whereas the latter requires validation to check that each fiber has a proposal.

A (proposalID, obCode) pair defines a unique target (ra/dec, requested total exposure time, instrument configuration, observing condition, etc.) and I think that adding these two keys would simplify tracking the observation progress for each target. eric, do you have any comments?

I'm also fine to add columns to the existing extension rather than adding a new one.

Could you please explain what obCode is?

Right, the first proposal makes sense to me too, it's consistent with what we did so far.

> eric, do you have any comments?

I think that would simplify things a lot for tracking queue progress if we can see the proposal and ob code directly from the op-db.

> Could you please explain what obCode is?

The ob_code is a string chosen by the PI for a particular OB (Observation Block), which is the "basic unit of observation" in the Subaru style of queue observing.  OB codes identify the OB being observed and must be unique within each proposal, thus the combination of proposal and ob_code is a unique OB (another observer might choose the same code).  The codes are chosen by the PI so that they may have meaningful names for planning their observations.

Here is a pull request for this ticket.

https://github.com/Subaru-PFS/datamodel/pull/118

As a combination of (proposalId, obCode) must be unique, some validation code must be present, but not yet implemented in the PR. In principle, this validation must be done when we ingest targets to the target database and the unique constraint has already been implemented there.

I'm adding a backward compatibility when proposalId and obCode do not exist in the case of previous pfsDesign and pfsConfig files. I'm adding "dummy" for these cases, but do you have any preferences for it? For example, proposalId = None and obCode = fiberId may also make sense. eric

> do you have any preferences for it? 

It may be nice to preserve the proposalId, IMO.  In such case we could then infer that an obCode == None is a classical observation, and the proposalId is valid. Just my .02c

We can't know which proposal ID is assigned for each fiber from a design/config file made, e.g., a year ago as there are no columns for these keys. That's why I thought we need to provide a dummy value.

I think in that case that None would be appropriate for proposalId.

Is there no other field for fiberId?  I think overloading a field with a value that is in a completely different domain will be potentially confusing.  These things then become more complicated to handle in software.

The current master branch of datamodel lists the following items in the DESIGN table (1st extension of pfsDesign file).

fiberId 32-bit int
?? catId 32-bit int??
?? tract 32-bit int??
?? patch string??
?? objId 64-bit int??
?? ra 64-bit float (degrees)??
?? dec 64-bit float (degrees)??
?? targetType 32-bit int (enumerated type: e.g. SCIENCE,SKY,FLUXSTD)??
?? fiberStatus 32-bit int (enumerated type: e.g. GOOD,BROKENFIBER,BLOCKED,BLACKSPOT)??
?? pfiNominal pair of 32-bit floats (millimeters on the PFI)??

The fiberId is already present, and I'm adding proposalId and obCode in the table.

After having a chat with eric , we agreed to put a dummy string ("N/A") rather than None to indicate the datatype explicitly. I'll make a change and make a commit. I guess I'll also need to be consistent in dummyCableB pfsDesign (and it should be straightforward).

Commits have been merged into master branches of datamodel, drp_stella, and pfs_utils.