https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/DAMD-144 Data Model <p>pfsConfig will now be written for each visit, instead of visit0 only.<br/> That also imply an opdb schema change.</p> DAMD-144

write pfsConfig per visitId not per visit0

Task Normal Done Done arnaud.lefur arnaud.lefur EngRun Wed, 23 Nov 2022 03:18:16 +0000 Mon, 12 Dec 2022 20:50:27 +0000 Mon, 12 Dec 2022 20:50:27 +0000 0 3 <p>It's not a trivial change, so before I'm going any further, I want to make sure, we're on the same page on this.<br/> It implies changes to datamodel, opdb probably some obs_pfs scripts as well and I'm also wondering about backward compatibility.<br/> My point is what do we actually gain ? Is it making someone's life easier <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a> <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=rhl" class="user-hover" rel="rhl">rhl</a> ?</p> <p>if we keep the current schema:</p> <ul> <li>we would have to create symlink for each sps visit taken with that config.</li> <li>also there is no direct mapping from pfs_visit_id to pfs_config entries, but adding a new table would work : <em>pfs_config_set : pfs_visit_id; visit0</em></li> </ul> <p>If we decide to change, my original though was to directly replace visit0 by visit but I think that's nuts actually.<br/> It would be better just to :</p> <ul> <li>add an optional visit argument to pfsConfig constructor, so visit=visit0 if not provided. Then the file name format becomes "pfsConfig-0x%016x-%06d.fits"%(self.pfsDesignId, self.visit) instead of "pfsConfig-0x%016x-%06d.fits"%(self.pfsDesignId, self.visit0)</li> <li>add a pfs_visit_id column in pfs_config table in opdb and put the unicity contraint on pfs_visit_id and not on visit0.</li> </ul> <p>I don't understand this.  Maybe we can chat this afternoon?</p> <p>After some discussions with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a>, we definitely eliminated the symlink option.<br/> I proposed an implementation for option 2, see datamodel : <a href="https://pfspipe.ipmu.jp/jira/browse/DAMD-144" title="write pfsConfig per visitId not per visit0" class="issue-link" data-issue-key="DAMD-144"><del>DAMD-144</del></a> :</p> <ul> <li>Add an optional visit argument to pfsConfig constructor, so visit=visit0 if not provided.</li> <li>File name format becomes "pfsConfig-0x%016x-%06d.fits"%(self.pfsDesignId, self.visit)</li> <li>Fps is the really the one which knows about visit0, it creates the config file, using the class method <em>fromPfsDesign(cls, pfsDesign, visit0, pfiCenter)</em></li> <li>After the convergence, IIC load this newly created pfsConfig in memory.</li> <li>Before taking a sps exposure with pfi, when it gets the visit, IIC create a new pfsConfig file using the new class method <em>fromPfsConfig(cls, pfsConfig, visit)</em></li> </ul> <p>pfsConfig files will be written to <em>/data/raw/$DATE/pfsConfig</em><br/> I think it should be simpler for drp ingest <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a> ?The exposure and the config files just needs to be copied over to the drp repo. <br/> As a bonus, it also solve the pfsConfig synchronisation issue to Hilo eg <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1718" title="copy fps.pfsConfig file from summit to hilo" class="issue-link" data-issue-key="INSTRM-1718"><del>INSTRM-1718</del></a></p> <p>I don't think DRP2D has ever really implemented the <tt>visit0</tt> concept. I'm fully in favour of dumping it and just using <tt>visit</tt>. I think that must be easier for everyone.</p> <p>Why are we not dropping <tt>visit0</tt> completely, in favour of <tt>visit</tt>? I think that gives us everything we need for pfsConfig. Maybe there are subtleties further upstream that I'm not aware of?</p> <p>Ack, just realised that I commented directly on the GitHub commits, rather than a PR. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=arnaud.lefur" class="user-hover" rel="arnaud.lefur">arnaud.lefur</a>, would you please create a PR?</p> <p>Made a couple of suggestions on the PR.</p> <p>I added a commit to drp&#95;stella to adjust to these changes, and the integration test now passes.</p> <p>Good. I'm ready to merge.</p> Relates INSTRM-1718 INSTRM-1802 Development Rank 0|zzsvrw: Reviewers price Sprint PreEngRun09Dec