https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-1582 Instrument control development <p>Kawanomoto-san developed the coefficients for the fiber focal plane using the same function as that used for the AG focal planes. Use this function for sky-pfi coordinate transformation. Also, the study using AG reveals that corrections of proper motion and equinox etc. is slightly different b/w Az-El and InR (in the telescope), so the correction is better to handle in pfs_utils.  I talked to Martin this point, and he agreed to call sky-pfi transformation using J2000.0 catalogue. Also he pointed out that unnecessary arguments needs to be deleted.</p> INSTRM-1582

Make sky-pfi coordinate transformation more consistent with AG

Task Normal Done Done yuki.moritani yuki.moritani EngRun Tue, 19 Apr 2022 01:48:55 +0000 Thu, 9 Jun 2022 15:36:59 +0000 Thu, 9 Jun 2022 15:36:59 +0000 pfs_utils 0 4 <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a> I'd like you to pass the proper motion and parallax of individual objects when you call the sky-pfi coordinate transformation, as follows.. is it possible? </p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> def CoordinateTransform(xyin, mode, pa=0.,                          cent=np.array([[0.], [0.]]),                          time=<span class="code-quote">'2020-01-01 10:00:00'</span>, pm=None, parallax=None): </pre> </div></div> <p>(pm should be the same shape of xyin, and parallax is an array of N elements.. although I'm putting None here)</p> <p> </p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a> I think this should work fine, and it would allow me to remove quite some code on my side, which is great!</p> <p>Is this new version already available somewhere, so that I can validate my adjustments on my side? I don't see it on <a href="https://github.com/Subaru-PFS/pfs_utils/" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/pfs_utils/</a> yet.</p> <p> </p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a> I'm sorry to be late.. but I just pushed the ticket branch to show the new I/F  among us:<br/>  <br/>  </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> def CoordinateTransform(xyin, mode, za=0., inr=0., pa=-90., adc=0.,                        cent=np.array([[0.], [0.]]), pm=None, par=None,                        time=<span class="code-quote">'2020-01-01 10:00:00'</span>, epoch=2015.5):  </pre> </div></div> <p>As we talked via email, I changed za to optional argument. Accordingly, I'd like <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=karr" class="user-hover" rel="karr">karr</a> to swap the second and third arguments to call the code. (<ins><tt>mode</tt> and <tt>za</tt> has been swapped.</ins>)<br/>  <br/> Although I wrote a little explanation as docstring,,, I'm writing a few sentences here just in case..<br/> You need to pass proper motion and parallax (I'll set zero if I don't have).<br/> The equinox of the catalogue is assumed J2000, and you should set reference epoch of the sky catalogue (default is 2015.5, following Gaia DR2.)<br/> Could you please check?<br/>  </p> <p>Thank you very much for this update! I have already started adjusting the calling code, and it seems to be very easy to do.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a> I pushed further modification to the ticket branch.</p> <p>I'm sorry for the last-minute work,,, but could you please check if it works on your side? </p> <p>Thank you! As far as I can tell, it continues to work.<br/> Perhaps there is a small typo in the documentation: Line 17 should possibly be<br/> <tt># sky_pfi : sky to F3C (using Kawanomoto's routine)</tt><br/> instead of<br/> <tt># sky_pfi_old : sky to F3C (using Kawanomoto's routine)</tt></p> <p>I have pushed the necessary changes to <a href="https://github.com/Subaru-PFS/ets_fiberalloc/tree/tickets/INSTRM-1582." class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_fiberalloc/tree/tickets/INSTRM-1582.</a> Unfortunately I could only test the <tt>demo_netflow.py</tt> script, since running <tt>subaru_fiber_allocation_2021nov.py</tt> requires the summit<img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/help_16.png" height="16" width="16" align="absmiddle" alt="" border="0"/> environment, which I cannot reproduce locally. Please notify me if you encounter any problems!</p> <p>Thank you martin. Could you tell whether there are any points to modify for <tt>shuffle</tt> as well?</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a> &amp; I are looking into this in the script but we got the following error:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> 2022-05-13T15:44:36.421 30 butler     newRun: cobracoach/20220513_039 /work/pfs/commissioning/netflow/lib/python3.8/site-packages/erfa/core.py:154: ErfaWarning: ERFA function <span class="code-quote">"pmsafe"</span> yielded 1 of <span class="code-quote">"distance overridden (Note 6)"</span>   warnings.warn(<span class="code-quote">'ERFA function <span class="code-quote">"{}"</span> yielded {}'</span>.format(func_name, wmsg), WARNING: Tried to get polar motions <span class="code-keyword">for</span> times after IERS data is valid. Defaulting to polar motion from the 50-yr mean <span class="code-keyword">for</span> those. This may affect precision at the arcsec level. Please check your astropy.utils.iers.conf.iers_auto_url and point it to a newer version <span class="code-keyword">if</span> necessary. [astropy.coordinates.builtin_frames.utils] 2022-05-13T15:44:37.163 30 astropy    Tried to get polar motions <span class="code-keyword">for</span> times after IERS data is valid. Defaulting to polar motion from the 50-yr mean <span class="code-keyword">for</span> those. This may affect precision at the arcsec level. Please check your astropy.utils.iers.conf.iers_auto_url and point it to a newer version <span class="code-keyword">if</span> necessary. WARNING: (some) times are outside of range covered by IERS table. Assuming UT1-UTC=0 <span class="code-keyword">for</span> coordinate transformations. [astropy.coordinates.builtin_frames.utils] 2022-05-13T15:44:37.166 30 astropy    (some) times are outside of range covered by IERS table. Assuming UT1-UTC=0 <span class="code-keyword">for</span> coordinate transformations. Traceback (most recent call last):   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/subaru_fiber_allocation.py"</span>, line 347, in &lt;module&gt;     main()   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/subaru_fiber_allocation.py"</span>, line 284, in main     vis, tp, tel, tgt, tgt_class_dict = nfutils.fiber_allocation(   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/pointing_utils/nfutils.py"</span>, line 295, in fiber_allocation     target_fppos = [tele.get_fp_positions(targets) <span class="code-keyword">for</span> tele in telescopes]   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/pointing_utils/nfutils.py"</span>, line 295, in &lt;listcomp&gt;     target_fppos = [tele.get_fp_positions(targets) <span class="code-keyword">for</span> tele in telescopes]   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/ets_fiberalloc/ets_fiber_assigner/netflow.py"</span>, line 550, in get_fp_positions     tmp = ctrans(xyin=tmp,   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/pfs_utils/python/pfs/utils/coordinates/CoordTransp.py"</span>, line 80, in CoordinateTransform     xyin, inr, za1 = convert_in_position(xyin, za, inr, pa, c,   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/pfs_utils/python/pfs/utils/coordinates/CoordTransp.py"</span>, line 266, in convert_in_position     az0, el0, inr = DCoeff.radec_to_subaru(ra0, dec0, pa, time,   File <span class="code-quote">"/work/pfs/commissioning/fiber_allocation/pfs_utils/python/pfs/utils/coordinates/DistortionCoefficients.py"</span>, line 660, in radec_to_subaru     paa = tel2.parallactic_angle(obs_time, coord4).deg   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astroplan/observer.py"</span>, line 534, in parallactic_angle     LST = time.sidereal_time(<span class="code-quote">'mean'</span>, longitude=self.location.lon)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 1903, in sidereal_time     <span class="code-keyword">return</span> self._sid_time_or_earth_rot_ang(longitude=longitude, **model_kwargs)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 1948, in _sid_time_or_earth_rot_ang     theta = self._call_erfa(function, scales)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 1970, in _call_erfa     erfa_parameters = [getattr(getattr(self, scale)._time, jd_part)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 1970, in &lt;listcomp&gt;     erfa_parameters = [getattr(getattr(self, scale)._time, jd_part)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 1336, in __getattr__     tm._set_scale(attr)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 584, in _set_scale     args.append(get_dt(jd1, jd2))   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py"</span>, line 2051, in _get_delta_ut1_utc     delta = iers_table.ut1_utc(jd1, jd2)   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/utils/iers/iers.py"</span>, line 281, in ut1_utc     <span class="code-keyword">return</span> self._interpolate(jd1, jd2, [<span class="code-quote">'UT1_UTC'</span>],   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/utils/iers/iers.py"</span>, line 414, in _interpolate     self._check_interpolate_indices(i1, i, np.max(mjd))   File <span class="code-quote">"/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/utils/iers/iers.py"</span>, line 354, in _check_interpolate_indices     raise IERSRangeError(<span class="code-quote">'(some) times are outside of range covered '</span> astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table. </pre> </div></div> <p>We have not been able to get solution yet.</p> <p>Does the value of "self._time" that is passed in netflow.py (line 552) look reasonable?</p> <p><cite>Thank you martin. Could you tell whether there are any points to modify for <tt>shuffle</tt> as well</cite>?</p> <p>I just pushed changes there as well. This should have no effect on the next commissioning run, since all the relevant parts of shuffle are now replicated in Onodera-san's script, and have been adjusted there already.</p> <p>??Does the value of "self._time" that is passed in netflow.py (line 552) look reasonable?<br/> I think we pass the time like this "2022-05-20T15:00:00Z". <br/> Strangely, the script works if we pass an earlier date like "2022-01-20T15:00:00Z"... </p> <p>I'm not sure it works to you, but if you run the code via notebook, could you try adding these lines before running the code?</p> <p> </p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> from astropy.utils.data <span class="code-keyword">import</span> clear_download_cache clear_download_cache() from astropy.utils <span class="code-keyword">import</span> iers # iers.IERS_A_URL iers.Conf.iers_auto_url.set(<span class="code-quote">'https:<span class="code-comment">//datacenter.iers.org/data/9/finals2000A.all'</span>) </span>from astroplan <span class="code-keyword">import</span> download_IERS_A download_IERS_A() </pre> </div></div> <p> </p> <p>OK, then this really looks like the tables in astropy need some kind of update. I was worried that the time string was getting scrambled somehow on my side, but this doesn't seem to be the case.</p> <p>Thank you, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a> . We already have actually changed the url to different sites and tested. The suggested lines actually didn't work unfortunately. Yes, it looks like the astropy problem.</p> <p>There was some discussion on astropy and they introduced a config parameter <tt>iers_degraded_accuracy</tt>.</p> <p><a href="https://docs.astropy.org/en/latest/utils/iers.html#configuration-parameters" class="external-link" rel="nofollow">https://docs.astropy.org/en/latest/utils/iers.html#configuration-parameters</a><br/> <a href="https://github.com/astropy/astropy/pull/13052" class="external-link" rel="nofollow">https://github.com/astropy/astropy/pull/13052</a></p> <p>I tried to install the latest version (i.e., the master branch) of astropy and set </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-python"> <span class="code-keyword">from</span> astropy.utils <span class="code-keyword">import</span> iers iers.conf.iers_degraded_accuracy = <span class="code-quote">"warn"</span> </pre> </div></div> <p>at the top of the script. Then, it looks working by switching to the IERS-B data. I'm not sure this is the right way, but I hope it is a reasonable workaround.</p> <p>Concerning the question how I can correct RA/Dec in a way that is consistent with pfs_utils: I think the relevant code snippet is <a href="https://github.com/Subaru-PFS/pfs_utils/blob/tickets/INSTRM-1582/python/pfs/utils/coordinates/DistortionCoefficients.py#L608-L633" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/pfs_utils/blob/tickets/INSTRM-1582/python/pfs/utils/coordinates/DistortionCoefficients.py#L608-L633</a></p> <p>I think I can copy this into my code, but if there is a preference to have this as a standalone helper function in pfs_utils, I'm happy to provide a PR for that!</p> <p>What do you think, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=rhl" class="user-hover" rel="rhl">rhl</a>?</p> <p>If needed, calculated ra-dec can be added as return of that routine.. so that you can call it.</p> <p>Adding</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> from astropy.utils <span class="code-keyword">import</span> iers iers.conf.auto_download = True </pre> </div></div> <p>works for me now.</p> <p>I don't know why because the default value is `True`.</p> <p>With this version, it seems that the relative geometry between AGs and cobras is bit strange. Attached is an example FoV map (on sky) with coordinates of assigned guide objects and science targets. </p> <p>(detailed command with Onodera-san's script is like this)</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> python $REPO/subaru_fiber_allocation.py --design_dir <span class="code-quote">"design/prep/yamashita"</span> --cobra_coach_dir <span class="code-quote">"cobracoach"</span>  --ra 268.70 --dec 20.27  --pa -90.0  --fluxstd_min_prob_f_star 0.0  --n_fluxstd 500  --guidestar_mag_max 20.0  --observation_time 2022-05-19T10:30:00Z  --conf config.toml --pfs_instdata_dir  /work/pfs/commissioning/fiber_allocation/pfs_instdata </pre> </div></div> <p>Changing PA doesn't change the relative geometry. I think this didn't happen at least in Nov., and I still don't know when this was changed. Or perhaps I did something stupid...</p> <p><span class="image-wrap" style=""><a id="15200_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15200/15200_pfsDesign-0x78e3b3034e015cf0_small.png" title="pfsDesign-0x78e3b3034e015cf0_small.png" file-preview-type="image" file-preview-id="15200" file-preview-title="pfsDesign-0x78e3b3034e015cf0_small.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15189/_thumb_15189.png" style="border: 0px solid black" /></a></span></p> <p>Let me check...</p> <p>Thank you <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=yuki.moritani" class="user-hover" rel="yuki.moritani">yuki.moritani</a> for your suggestion offline.</p> <p>OK, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=monodera" class="user-hover" rel="monodera">monodera</a> : Is the script pass the same PA to both netflow part and guide star selection part?</p> <p>According to the code, it looks yes.</p> <p>Then, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=Martin.Reinecke" class="user-hover" rel="Martin.Reinecke">Martin Reinecke</a> : Are the netflow (or nf.Telescope) and the guide star part using the same PA?</p> <p>The guide positions should be around sides of the hexagon (not <em>vertices).</em> See the movie in the middle of the official website (<a href="https://pfs.ipmu.jp/instrumentation.html)" class="external-link" rel="nofollow">https://pfs.ipmu.jp/instrumentation.html</a>). Sorry but I cannot quickly show a better photo.</p> <p><cite>If needed, calculated ra-dec can be added as return of that routine.. so that you can call it.</cite></p> <p>Do you mean by introducing a new transform mode, e.g. "sky_to_corrected_sky"? That would be a great addition!</p> <p><cite>Then, Martin Reinecke : Are the netflow (or nf.Telescope) and the guide star part using the same PA</cite>?</p> <p>The "guidecam_geometry" function is independent of the PA (as far as I understand), since it works entirely with local PFI coordinates. All the remaining transforms are probably taking part inside the "subaru_fiber_allocation.py" script. If you can point me to the location of this script, I'm happy to look more closely. (I assume it has evolved from "subaru_fiber_allocation_2021nov.py", but I don't know where it is located.)</p> <p>Hi, the script for this commissioning run can be found here (commissioning_2022may branch of ets_target_database).</p> <p><a href="https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2022may/examples/commissioning_2022may" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2022may/examples/commissioning_2022may</a></p> <p>I think I was able to fix the issue following the advice by Yabe-san and Martin. I found a bug in my script. Thanks a lot.</p> <p>I attached some example pfsDesigns by using the following command.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-bash"> python ./subaru_fiber_allocation.py \ --conf ../../../database_configs/config_pfsa-db01-gb_commissioning_2022may.toml \ --ra 161.72486 \ --dec 28.53286 \ --pa -90.0 \ --cobra_coach_dir coach \ --design_dir=design \ --n_fluxstd 0 \ --fluxstd_min_prob_f_star 0.0 \ --guidestar_mag_max 20.0 \ --observation_time 2022-05-19T10:30:00Z \ --exptime 900 \ --raster_scan \ --raster_mag_max 20 </pre> </div></div> <p>Then by setting --sm option to specify which spectrographs to be used, one can limit cobras to be allocated to objects.</p> <p> <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15198/15198_pfsDesign-0x78e3b3034e015cf0_sm1%2B2%2B3%2B4.pdf" title="pfsDesign-0x78e3b3034e015cf0_sm1+2+3+4.pdf attached to INSTRM-1582">pfsDesign-0x78e3b3034e015cf0_sm1+2+3+4.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> : All spectrographs<br/> <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15199/15199_pfsDesign-0x413bcf208d3daa22_sm1%2B3.pdf" title="pfsDesign-0x413bcf208d3daa22_sm1+3.pdf attached to INSTRM-1582">pfsDesign-0x413bcf208d3daa22_sm1+3.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> : SM1 and SM3<br/> <span class="nobr"><a href="https://pfspipe.ipmu.jp/jira/secure/attachment/15197/15197_pfsDesign-0x6b7813a6e0eb4ef9_sm1.pdf" title="pfsDesign-0x6b7813a6e0eb4ef9_sm1.pdf attached to INSTRM-1582">pfsDesign-0x6b7813a6e0eb4ef9_sm1.pdf^{<img class="rendericon" src="https://pfspipe.ipmu.jp/jira/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> : SM1</p> <p>In the figures, a large red circle indicates the location of a calibration star provided by Yamashita-san and you can see it is allocated to SM1. Other allocated objects are stars used for the raster scan test. </p> <p>AG star locations look also okay.</p> <p> <span class="image-wrap" style=""><a id="15200_thumb" href="https://pfspipe.ipmu.jp/jira/secure/attachment/15200/15200_pfsDesign-0x78e3b3034e015cf0_small.png" title="pfsDesign-0x78e3b3034e015cf0_small.png" file-preview-type="image" file-preview-id="15200" file-preview-title="pfsDesign-0x78e3b3034e015cf0_small.png"><img src="https://pfspipe.ipmu.jp/jira/secure/thumbnail/15200/_thumb_15200.png" style="border: 0px solid black" /></a></span> </p> <p>It was tested and used by a few colleagues last run. As it's been merged to master, I close this ticket.</p> Relates INSTRM-1609 Development Rank 0|02qpjd:00xzns Sprint PreEngRun05 F PreEngRun05 G