https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/OBSPROC-21 PFS observation processing/procedure <p>As we discussed in the last ICS/PFI-MCS telecon, we want to prepare a script to generate pfsDesign with both targets and guide stars for the coming runs. For the previous runs, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a> prepared commisioning.py in ETS package and then <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a> modified the code and prepared subaru_fiber_allocation_2021nov.py to talk to GaiaDB and targetDB etc. For the next run, we should have an official script to make pfsDesigns to observe in the similar way. We may be able to merge Onodera-san's code to ETS package if possible.</p> OBSPROC-21

Official script to prepare pfsDesign file connecting DBs including targetDB

Story Major Done Done monodera Kiyoto Yabe EngRun Thu, 14 Apr 2022 00:48:13 +0000 Wed, 8 Jun 2022 01:02:25 +0000 Wed, 8 Jun 2022 00:54:24 +0000 ets_fiberalloc ets_target_database 0 8 <p>I'd be very happy to merge the changes! Is it possible to have the modified version committed to a new branch on <a href="https://github.com/Subaru-PFS/ets_fiberalloc," class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_fiberalloc,</a> so that we can look at the differences easily?</p> <p> </p> <p>OK. <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a>, could you update subaru_fiber_allocation_2021nov.py (if needed) and push to a new branch of ets_fiberalloc? </p> <p>I have pushed a new branch (tickets/<a href="https://pfspipe.ipmu.jp/jira/browse/OBSPROC-21" title="Official script to prepare pfsDesign file connecting DBs including targetDB" class="issue-link" data-issue-key="OBSPROC-21"><del>OBSPROC-21</del></a>/monodera) to the repository.</p> <p><a href="https://github.com/Subaru-PFS/ets_fiberalloc/tree/tickets/OBSPROC-21/monodera" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_fiberalloc/tree/tickets/OBSPROC-21/monodera</a></p> <p>Could you please check if the pushed script and readme file is okay?</p> <p>Thank you very much for providing the script and documentation! I think this is a great impovement over `commissioning.py` (which was originally meant for demonstration purposes only!) and should be preferred to it in the future.</p> <p>Related to this I have one technical question, perhaps mainly to <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a>: should we continue to host this file inside the fiber allocation package? I think it is of very broad interest and probably deserves a separate repository. I'm of course very happy to keep it in ets_fiberalloc, but not many people will expect it in there.</p> <p> </p> <p>Hi Martin, I agree that <tt>commissioning.py</tt> probably should be in a separate repository, although it's not obvious at this moment which one. I will discuss this with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a>. In the meantime, let's keep it in <tt>ets_fiberalloc</tt>.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=wanqqq31" class="user-hover" rel="wanqqq31">Wanqiu He</a> is developing the pointing optimizer and there is a part to allocate fibers by calling netflow. I think it would be great to make a repository to integrate them and work together.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a> I have put a simplified version of PPP to the repository:<br/> <a href="https://github.com/Subaru-PFS/ets_pointing/tree/wanqqq-test/PPP" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_pointing/tree/wanqqq-test/PPP</a></p> <p>The "netflow" part of the script may be the one you want to check. <img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Regarding Hasan's comment on repository</p> <blockquote><p>Hi Martin, I agree that commissioning.py probably should be in a separate repository, although it's not obvious at this moment which one. I will discuss this with <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a>. In the meantime, let's keep it in ets_fiberalloc.</p></blockquote> <p>Hassan and I talked about it. We think it nicer to have <tt>commissioning.py</tt> on{{ ets_pointing}} repository, rather than <tt>ets_fiberalloc</tt> ...Also it is nice to rename .py to more proper one. What o you think?</p> <p>I'm updating a script to generate pfsDesign files. I'm trying to split functions so that we can port it into the ets_pointing repository. The code can be found in the commissioning_2022may branch in the ets_target_database repository.</p> <p><a href="https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2022may" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2022may</a></p> <p>Note that the README is not updated (but should be more or less similar) and will be done by the commissioning run.</p> <p>In the previous commissioning.py script, there is a getBench function. I'm wondering if we still need it or not. In the script provided by <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=wanqqq31" class="user-hover" rel="wanqqq31">Wanqiu He</a>, it uses bench = Bench(layout="full") with no additional adjustment.</p> <p>This is a great change! The goal was to remove all the individual fixes in getBench() over time, and if this is indeed now possible, I'm very happy.</p> <p>Did you get some error before? I haven't tried to remove getBench() at all, so I don't know if it works properly or not. Also, cobraCoach may be anyway needed for the collision check? </p> <p>Yes, there were errors before, e.g. "negative link length detected".</p> <p>Sorry, I think that were actually link lengths of zero, or impossibly long links.</p> <p>Thanks. Then, it looks promising.</p> <p>While making the adjustments necessary for <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-1582" title="Make sky-pfi coordinate transformation more consistent with AG" class="issue-link" data-issue-key="INSTRM-1582"><del>INSTRM-1582</del></a> I wondered whether we need to take proper motion and parallax into account for the fiber targets. So far we only use PM information when determining the exact position of the guide star candidates.<br/> If we need this for the science targets, I expect we have to enhance the current target file format by a few extra columns.<br/> What do you think, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a>?</p> <p>I think this is an excellent idea.</p> <p>The attached file has a function</p> <p>"update_coords_for_proper_motion"</p> <p>that may be helpflul.</p> <p><span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15179/15179_do_shuffle_target.py" title="do_shuffle_target.py attached to OBSPROC-21">do_shuffle_target.py^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span></p> <p>We have this code already in the "ets_shuffle.convenience" module. But if I understand correctly, this task will actually be done for us by the updated coordinate transforms in "pfs_utils", so we can probably remove it. Still, we have to tell "pfs_utils" the proper motion of our targets, which we currently don't know (since it is not listed in the target files).</p> <p>As it is natural to include proper motions for Galactic science as well as flux standard stars, it would be nice to have them in the table. I guess just adding extra columns shouldn't be a problem. I implemented pm info in the targetDB with the default values of zero for (pmra, pmdec, and parallax). </p> <p>Slightly off topic, but related: In the Gaia catalogs, epoch is defined as a float (2015.5 for DR2), while epoch in the PFS datamodel is defined as a 7-character string like "J2015.5". The conversion is straightforward, but looks redundant as users convert floats to strings at Phase 2, the observatory store them as strings in the DB, and the codes convert strings to floats / floats to strings (perhaps multiple times). </p> <p>Maybe targetDB (and upstream DBs) can use epoch in float (only accepts Julian epochs) and convert it to strings to conform the datamodel when writing to the pfsDesign. </p> <p>BTW, a more comprehensive example of proper motion (+distance) correction can be found here, though it does not seem to be relevant for us.</p> <p><a href="https://docs.astropy.org/en/stable/coordinates/apply_space_motion.html" class="external-link" rel="nofollow">https://docs.astropy.org/en/stable/coordinates/apply_space_motion.html</a></p> <p>I'm wondering whether proper motions need to be corrected before fiber assignment in order to take dots and fiber collisions into account. If pfs_utils takes care of it before writing to pfsDesign after fiber assignment, should input coordinates be the original ones before the correction? I think as long as the epoch is set and sent properly to pfs_utils, the correction can be before the fiber assignment. In any case, if there is a single function in the pfs_utils for the pm correction would be nice to avoid unnecessary redundancy.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=hassan" class="user-hover" rel="hassan">hassan</a> <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15182/15182_pfsDesign-0x66252be36b4c117a.fits" title="pfsDesign-0x66252be36b4c117a.fits attached to OBSPROC-21">pfsDesign-0x66252be36b4c117a.fits^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> is an example from the current code. There are many unassigned fibers mainly due to the lack of enough science targets. </p> <p>I used the following parameters.<br/> ra = 215.0<br/> dec = 43.0<br/> pa = 0.0<br/> obstime = 2022-05-20T15:00:00Z</p> <p>Thank you. I'll do a double check.</p> <p>Thanks. I will also have a look.</p> <p><cite>I'm wondering whether proper motions need to be corrected before fiber assignment in order to take dots and fiber collisions into account. If pfs_utils takes care of it before writing to pfsDesign after fiber assignment, should input coordinates be the original ones before the correction? I think as long as the epoch is set and sent properly to pfs_utils, the correction can be before the fiber assignment. In any case, if there is a single function in the pfs_utils for the pm correction would be nice to avoid unnecessary redundancy.</cite></p> <p>My understanding is the following:</p> <ul class="alternate" type="square"> <li>up to now, if I wanted the exact position of a target in the focal plane, I had to do all necessary corrections beforehand and then call the coordinate transform from pfs.utils on the resulting RA/Dec.</li> <li>in the future, I will simply pass the catalog RA/Dec, epoch, pm, parallax and observation time to the transform routine and get the correct focal plane locations back.</li> </ul> <p>In other words I will never even see the corrected RA/Dec values (which is fine, since the fiber assigner only works with focal plane coordinates). That of course leaves the question which sky coordinates should be written into the pfs design file: the catalog ones (which could also be found in the catalog) or the corrected ones (which I don't have any more with the new approach).</p> <p>I fear that my understanding of the pfs design file is quite insufficient and therefore some of the data in there feel redundant to me. If the object and catalog ID of a target are known, as well as the observation time, then RA/Dec can in principle be obtained on the fly.</p> <p>From today's ICS/PFI telecon: need to do a tweak from final RA/DEC to final observed x/y. Design would have nominal RA/DEC. Config will have final RA/DEC taking into account airmass. The SHA for the Config should not be recalculated from the updated RA/DEC. </p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a> <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=kiyoto.yabe" class="user-hover" rel="kiyoto.yabe">Kiyoto Yabe</a>: are we in a position to close this ticket now, given what is already implemented? We can create new tickets to capture additional work. </p> <p>Yes, I guess we can merge `commissioning_2022may` branch and close this ticket. We can open `commissioning_2022jun` ticket if needed (probably we need).</p> <p>I have just merged the commissioning2022may branch to the main in the ets_target_database repository. I agree to close the ticket now.</p> <p>We will still need to discuss where the scripts to be hosted, but can be discussed in a separate ticket.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a>: I think Yuki suggested <tt>ets_pointing</tt> as the git repo location for these scripts (<a href="https://pfspipe.ipmu.jp/jira/browse/OBSPROC-21?focusedCommentId=30794&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-30794" class="external-link" rel="nofollow">https://pfspipe.ipmu.jp/jira/browse/OBSPROC-21?focusedCommentId=30794&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-30794</a>). Or is your question related to something else? In any case, that can be captured in a separate ticket as you say.</p> <p>I forgot about that comment. I agree with her.</p> <p>I close this ticket as we have agreed and will open another ticket about the script location.</p> Relates OBSPROC-25 Development Rank 0|zzstqm:zzwi Sprint PreEngRun05 G PreEngRun06June