[DAMD-144] write pfsConfig per visitId not per visit0 Created: 23/Nov/22  Updated: 13/Dec/22  Resolved: 13/Dec/22

Status: Done
Project: Data Model
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: arnaud.lefur Assignee: arnaud.lefur
Resolution: Done Votes: 0
Labels: EngRun
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to INSTRM-1718 copy fps.pfsConfig file from summit t... Won't Fix
relates to INSTRM-1802 write actual pfsConfig for each sps e... Done
Sprint: PreEngRun09Dec
Reviewers: price

 Description   

pfsConfig will now be written for each visit, instead of visit0 only.
That also imply an opdb schema change.



 Comments   
Comment by arnaud.lefur [ 01/Dec/22 ]

It's not a trivial change, so before I'm going any further, I want to make sure, we're on the same page on this.
It implies changes to datamodel, opdb probably some obs_pfs scripts as well and I'm also wondering about backward compatibility.
My point is what do we actually gain ? Is it making someone's life easier price cloomis rhl ?

if we keep the current schema:

  • we would have to create symlink for each sps visit taken with that config.
  • also there is no direct mapping from pfs_visit_id to pfs_config entries, but adding a new table would work : pfs_config_set : pfs_visit_id; visit0

If we decide to change, my original though was to directly replace visit0 by visit but I think that's nuts actually.
It would be better just to :

  • add an optional visit argument to pfsConfig constructor, so visit=visit0 if not provided. Then the file name format becomes "pfsConfig-0x%016x-%06d.fits"%(self.pfsDesignId, self.visit) instead of "pfsConfig-0x%016x-%06d.fits"%(self.pfsDesignId, self.visit0)
  • add a pfs_visit_id column in pfs_config table in opdb and put the unicity contraint on pfs_visit_id and not on visit0.
Comment by rhl [ 01/Dec/22 ]

I don't understand this.  Maybe we can chat this afternoon?

Comment by arnaud.lefur [ 02/Dec/22 ]

After some discussions with cloomis, we definitely eliminated the symlink option.
I proposed an implementation for option 2, see datamodel : DAMD-144 :

  • Add an optional visit argument to pfsConfig constructor, so visit=visit0 if not provided.
  • File name format becomes "pfsConfig-0x%016x-%06d.fits"%(self.pfsDesignId, self.visit)
  • Fps is the really the one which knows about visit0, it creates the config file, using the class method fromPfsDesign(cls, pfsDesign, visit0, pfiCenter)
  • After the convergence, IIC load this newly created pfsConfig in memory.
  • Before taking a sps exposure with pfi, when it gets the visit, IIC create a new pfsConfig file using the new class method fromPfsConfig(cls, pfsConfig, visit)

pfsConfig files will be written to /data/raw/$DATE/pfsConfig
I think it should be simpler for drp ingest price ?The exposure and the config files just needs to be copied over to the drp repo.
As a bonus, it also solve the pfsConfig synchronisation issue to Hilo eg INSTRM-1718

Comment by price [ 02/Dec/22 ]

I don't think DRP2D has ever really implemented the visit0 concept. I'm fully in favour of dumping it and just using visit. I think that must be easier for everyone.

Comment by price [ 03/Dec/22 ]

Why are we not dropping visit0 completely, in favour of visit? I think that gives us everything we need for pfsConfig. Maybe there are subtleties further upstream that I'm not aware of?

Comment by price [ 03/Dec/22 ]

Ack, just realised that I commented directly on the GitHub commits, rather than a PR. arnaud.lefur, would you please create a PR?

Comment by price [ 08/Dec/22 ]

Made a couple of suggestions on the PR.

Comment by price [ 13/Dec/22 ]

I added a commit to drp_stella to adjust to these changes, and the integration test now passes.

Comment by arnaud.lefur [ 13/Dec/22 ]

Good. I'm ready to merge.

Generated at Sat Feb 10 15:34:43 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.