https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/OBSPROC-15 PFS observation processing/procedure <p>By looking at some previous codes to create a pfsDesign file (soon we will have a common code, I suppose, though), we need to have a code to prepare pfsDesign by talking to targetDB, He-san's code, and fiber allocation softwares, etc.</p> OBSPROC-15

Prepare a code to make pfsDesign

Task Normal Done Done monodera monodera EngRun Tue, 26 Oct 2021 06:18:22 +0000 Thu, 25 Nov 2021 01:25:29 +0000 Thu, 25 Nov 2021 01:25:28 +0000 0 9 <p><a href="https://github.com/Subaru-PFS/ets_fiberalloc/blob/commissioning_demo/commissioning.py" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_fiberalloc/blob/commissioning_demo/commissioning.py</a>  is probably a very good starting point for the coming November run.</p> <p>Minimum working modification is made (see <tt>gen_target_list_from_targetdb()</tt>). <br/> <a href="https://github.com/Subaru-PFS/ets_target_database/blob/commissioning_2021nov/examples/commissioning_2021nov/commissioning.py" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_target_database/blob/commissioning_2021nov/examples/commissioning_2021nov/commissioning.py</a></p> <p>The script still needs to be reviewed and tested. </p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a>points out that the pfsDesignId is calculated from the desired positions of the cobras, and that this isn't available until the last stage in the construction of the pfsDesign file (starting with boresight and rotation, adding the guide stars, and then finally the cobra positions). In consequence, he is forced to assign his own "pfsDesignId" rather than allowing the PFS code to calculate it.<br/> This is a result of using pfsDesign files in ways that I didn't consider when writing down the datamodel, but which make total sense. One option would be to <em>not</em> use pfsDesign files in this way, but I think that that's probably not the best solution as we'd need to define a new file format, and it'd still need a unique name. Another is to proceed using an external generation of the pfsDesignId, in which case we need to be sure that they are indeed unique. Alternatively, we could extend the definition of the pfsDesignId to include the (nominal) boresight, rotation, and list of guide star positions. </p> <p>I favour the last of these proposals (which will require a DAMD ticket). <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a>? <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=price" class="user-hover" rel="price">price</a>?</p> <p>That seems clean, and at this point in time safe.</p> <p>So for the last proposal, would Martin need, in addition to the (nominal) boresight, rotation, list of guidestar positions, to assume default positions for the cobras (presumably the home positions)?</p> <p>I think we (now) handle <tt>NaN</tt> correctly, so this should be automatic.</p> <p>I don't like the idea of building up a <tt>pfsDesign</tt> file in fits and starts, as I think it's bound to cause confusion. I suggest we add a new file type, say <tt>pfsPlan</tt>, which will be similar to <tt>pfsDesign</tt> but without the cobra positions, and have the <tt>pfsPlanId</tt> hash calculated from the available information.</p> <p>The implemented <tt>PfsDesign</tt>/<tt>PfsConfig</tt> hierarchy already supports building on top of existing classes, so it should be simple to code.</p> <p>If we go the route of introducing new file types, this makes things a bit more complicated for `ets_shuffle` and `ets_fiberalloc`...</p> <p> </p> <p>Currently the idea is that</p> <ul> <li>`ets_shuffle` reads boresight and PA from a "pfsDesign-like" file and outputs the input file, enriched with guide star information.</li> <li>`ets_fiberalloc` reads boresight and PA from a "pfsDesign-like" file and outputs the input file, enriched by cobra-target assignment information.</li> </ul> <p> </p> <p>This has the nice property that the order of execution doesn't matter for those two programs, and that they just use PFSdesign files (in various stages of construction) fo input and output. I'd be happy not to lose this kind of simplicity, especially since it is fairly easy to test a PFSDesign for completeness when this is really required.</p> <p> </p> <p>From <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a> in the zoom chat during the commissioning telecon (Nov 10/11, 2021):</p> <p><cite>The most up to date version of the commissioning.py script is at <a href="https://github.com/Subaru-PFS/ets_fiberalloc/blob/commissioning_demo_2/commissioning.py" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_fiberalloc/blob/commissioning_demo_2/commissioning.py</a></cite></p> <p>BTW, by setting <tt>–use_gurobi</tt> as follows, you don't need to put <tt>True</tt> or <tt>False</tt> explicitly.</p> <p><tt>parser.add_argument("--use_gurobi", action="store_true", help="use Gurobi solver instead of PuLP")</tt></p> <p>Thank you, this is very useful! I will update the code accordingly. (My main language is C++ <img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/>)</p> <p>Just a small thing but the current commissioning.py didn't work for me:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Traceback (most recent call last): File <span class="code-quote">"commissioning.py"</span>, line 393, in &lt;module&gt; main() File <span class="code-quote">"commissioning.py"</span>, line 382, in main args = get_arguments() File <span class="code-quote">"commissioning.py"</span>, line 88, in get_arguments parser.add_argument(<span class="code-quote">"--use_gurobi"</span>, type=bool, action=<span class="code-quote">'store_true'</span>, help=<span class="code-quote">"use Gurobi solver instead of PuLP"</span>) File <span class="code-quote">"/opt/anaconda3/lib/python3.7/argparse.py"</span>, line 1359, in add_argument action = action_class(**kwargs) TypeError: __init__() got an unexpected keyword argument <span class="code-quote">'type'</span> } </pre> </div></div> <p>Please remove `type=bool`.</p> <p><tt>type=bool</tt> is not needed. I don't know how to merge the scripts of yours and mine. What does work best for us? I'm trying to untouch the assignment part so that integration can be easy. But I want to put more parameters such as flues in pfsDesign and will need to write a separate version of <tt>generatePfsDesign()</tt>.</p> <p>I removed the `type=bool`. Strangely, it even worked for me when the argument was present.</p> <p>Eventually, I believe that I was able to manage to create a pfsDesign file by updating the Martin's commissioning.py. My code (subaru_fiber_allocation_2021nov.py) and README are found at <a href="https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2021nov/examples/commissioning_2021nov" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2021nov/examples/commissioning_2021nov</a> .</p> <p>I attach the pfsDesign file generated with the example in the README above. If someone can check if it looks okay and suggestion for further tests of the script with various conditions, I'd appreciate.</p> <p> <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/14487/14487_pfsDesign-0x6a119411f0b96b28.fits" title="pfsDesign-0x6a119411f0b96b28.fits attached to OBSPROC-15">pfsDesign-0x6a119411f0b96b28.fits^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> </p> <p>I think I can close it now as the code was developed for the engineering run and tested by a few people, but the fiber configuration was not tested due to the AG and rotator issues.</p> Development Epic Link OBSPROC-1 Rank 0|zzs2x9: Sprint PreEngRun4 Story Points 4.0