[INSTRM-1582] Make sky-pfi coordinate transformation more consistent with AG Created: 19/Apr/22  Updated: 10/Jun/22  Resolved: 10/Jun/22

Status: Done
Project: Instrument control development
Component/s: pfs_utils
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: yuki.moritani Assignee: yuki.moritani
Resolution: Done Votes: 0
Labels: EngRun
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PDF File pfsDesign-0x413bcf208d3daa22_sm1+3.pdf     PDF File pfsDesign-0x6b7813a6e0eb4ef9_sm1.pdf     PDF File pfsDesign-0x78e3b3034e015cf0_sm1+2+3+4.pdf     PNG File pfsDesign-0x78e3b3034e015cf0_small.png     PNG File test.png    
Issue Links:
Relates
relates to INSTRM-1609 Calculate fibre positions in mm just ... Done
Sprint: PreEngRun05 F, PreEngRun05 G

 Description   

Kawanomoto-san developed the coefficients for the fiber focal plane using the same function as that used for the AG focal planes. Use this function for sky-pfi coordinate transformation. Also, the study using AG reveals that corrections of proper motion and equinox etc. is slightly different b/w Az-El and InR (in the telescope), so the correction is better to handle in pfs_utils.  I talked to Martin this point, and he agreed to call sky-pfi transformation using J2000.0 catalogue. Also he pointed out that unnecessary arguments needs to be deleted.

 Comments   
Comment by yuki.moritani [ 20/Apr/22 ]

Martin Reinecke I'd like you to pass the proper motion and parallax of individual objects when you call the sky-pfi coordinate transformation, as follows.. is it possible? 

 

def CoordinateTransform(xyin, mode, pa=0.,
                         cent=np.array([[0.], [0.]]),
                         time='2020-01-01 10:00:00',
                         pm=None, parallax=None):

(pm should be the same shape of xyin, and parallax is an array of N elements.. although I'm putting None here)

 

Comment by Martin Reinecke [ 20/Apr/22 ]

yuki.moritani I think this should work fine, and it would allow me to remove quite some code on my side, which is great!

Is this new version already available somewhere, so that I can validate my adjustments on my side? I don't see it on https://github.com/Subaru-PFS/pfs_utils/ yet.

 

Comment by yuki.moritani [ 06/May/22 ]

Martin Reinecke I'm sorry to be late.. but I just pushed the ticket branch to show the new I/F  among us:
 
 

def CoordinateTransform(xyin, mode, za=0., inr=0., pa=-90., adc=0.,
                        cent=np.array([[0.], [0.]]), pm=None, par=None,
                        time='2020-01-01 10:00:00', epoch=2015.5):  

As we talked via email, I changed za to optional argument. Accordingly, I'd like karr to swap the second and third arguments to call the code. (mode and za has been swapped.)
 
Although I wrote a little explanation as docstring,,, I'm writing a few sentences here just in case..
You need to pass proper motion and parallax (I'll set zero if I don't have).
The equinox of the catalogue is assumed J2000, and you should set reference epoch of the sky catalogue (default is 2015.5, following Gaia DR2.)
Could you please check?
 

Comment by Martin Reinecke [ 06/May/22 ]

Thank you very much for this update! I have already started adjusting the calling code, and it seems to be very easy to do.

Comment by yuki.moritani [ 12/May/22 ]

Martin Reinecke I pushed further modification to the ticket branch.

I'm sorry for the last-minute work,,, but could you please check if it works on your side? 

Comment by Martin Reinecke [ 13/May/22 ]

Thank you! As far as I can tell, it continues to work.
Perhaps there is a small typo in the documentation: Line 17 should possibly be
# sky_pfi : sky to F3C (using Kawanomoto's routine)
instead of
# sky_pfi_old : sky to F3C (using Kawanomoto's routine)

Comment by Martin Reinecke [ 13/May/22 ]

I have pushed the necessary changes to https://github.com/Subaru-PFS/ets_fiberalloc/tree/tickets/INSTRM-1582. Unfortunately I could only test the demo_netflow.py script, since running subaru_fiber_allocation_2021nov.py requires the summit environment, which I cannot reproduce locally. Please notify me if you encounter any problems!

Comment by yuki.moritani [ 14/May/22 ]

Thank you martin. Could you tell whether there are any points to modify for shuffle as well?

Comment by Kiyoto Yabe [ 14/May/22 ]

monodera & I are looking into this in the script but we got the following error:

2022-05-13T15:44:36.421 30 butler     newRun: cobracoach/20220513_039
/work/pfs/commissioning/netflow/lib/python3.8/site-packages/erfa/core.py:154: ErfaWarning: ERFA function "pmsafe" yielded 1 of "distance overridden (Note 6)"
  warnings.warn('ERFA function "{}" yielded {}'.format(func_name, wmsg),
WARNING: Tried to get polar motions for times after IERS data is valid. Defaulting to polar motion from the 50-yr mean for those. This may affect precision at the arcsec level. Please check your astropy.utils.iers.conf.iers_auto_url and point it to a newer version if necessary. [astropy.coordinates.builtin_frames.utils]
2022-05-13T15:44:37.163 30 astropy    Tried to get polar motions for times after IERS data is valid. Defaulting to polar motion from the 50-yr mean for those. This may affect precision at the arcsec level. Please check your astropy.utils.iers.conf.iers_auto_url and point it to a newer version if necessary.
WARNING: (some) times are outside of range covered by IERS table. Assuming UT1-UTC=0 for coordinate transformations. [astropy.coordinates.builtin_frames.utils]
2022-05-13T15:44:37.166 30 astropy    (some) times are outside of range covered by IERS table. Assuming UT1-UTC=0 for coordinate transformations.
Traceback (most recent call last):
  File "/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/subaru_fiber_allocation.py", line 347, in <module>
    main()
  File "/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/subaru_fiber_allocation.py", line 284, in main
    vis, tp, tel, tgt, tgt_class_dict = nfutils.fiber_allocation(
  File "/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/pointing_utils/nfutils.py", line 295, in fiber_allocation
    target_fppos = [tele.get_fp_positions(targets) for tele in telescopes]
  File "/work/pfs/commissioning/fiber_allocation/ets_target_database/examples/commissioning_2022may/pointing_utils/nfutils.py", line 295, in <listcomp>
    target_fppos = [tele.get_fp_positions(targets) for tele in telescopes]
  File "/work/pfs/commissioning/fiber_allocation/ets_fiberalloc/ets_fiber_assigner/netflow.py", line 550, in get_fp_positions
    tmp = ctrans(xyin=tmp,
  File "/work/pfs/commissioning/fiber_allocation/pfs_utils/python/pfs/utils/coordinates/CoordTransp.py", line 80, in CoordinateTransform
    xyin, inr, za1 = convert_in_position(xyin, za, inr, pa, c, 
  File "/work/pfs/commissioning/fiber_allocation/pfs_utils/python/pfs/utils/coordinates/CoordTransp.py", line 266, in convert_in_position
    az0, el0, inr = DCoeff.radec_to_subaru(ra0, dec0, pa, time, 
  File "/work/pfs/commissioning/fiber_allocation/pfs_utils/python/pfs/utils/coordinates/DistortionCoefficients.py", line 660, in radec_to_subaru
    paa = tel2.parallactic_angle(obs_time, coord4).deg
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astroplan/observer.py", line 534, in parallactic_angle
    LST = time.sidereal_time('mean', longitude=self.location.lon)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 1903, in sidereal_time
    return self._sid_time_or_earth_rot_ang(longitude=longitude, **model_kwargs)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 1948, in _sid_time_or_earth_rot_ang
    theta = self._call_erfa(function, scales)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 1970, in _call_erfa
    erfa_parameters = [getattr(getattr(self, scale)._time, jd_part)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 1970, in <listcomp>
    erfa_parameters = [getattr(getattr(self, scale)._time, jd_part)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 1336, in __getattr__
    tm._set_scale(attr)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 584, in _set_scale
    args.append(get_dt(jd1, jd2))
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/time/core.py", line 2051, in _get_delta_ut1_utc
    delta = iers_table.ut1_utc(jd1, jd2)
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/utils/iers/iers.py", line 281, in ut1_utc
    return self._interpolate(jd1, jd2, ['UT1_UTC'],
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/utils/iers/iers.py", line 414, in _interpolate
    self._check_interpolate_indices(i1, i, np.max(mjd))
  File "/work/pfs/commissioning/netflow/lib/python3.8/site-packages/astropy/utils/iers/iers.py", line 354, in _check_interpolate_indices
    raise IERSRangeError('(some) times are outside of range covered '
astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table.

We have not been able to get solution yet.

Comment by Martin Reinecke [ 14/May/22 ]

Does the value of "self._time" that is passed in netflow.py (line 552) look reasonable?

Comment by Martin Reinecke [ 14/May/22 ]

Thank you martin. Could you tell whether there are any points to modify for shuffle as well?

I just pushed changes there as well. This should have no effect on the next commissioning run, since all the relevant parts of shuffle are now replicated in Onodera-san's script, and have been adjusted there already.

Comment by Kiyoto Yabe [ 14/May/22 ]

??Does the value of "self._time" that is passed in netflow.py (line 552) look reasonable?
I think we pass the time like this "2022-05-20T15:00:00Z". 
Strangely, the script works if we pass an earlier date like "2022-01-20T15:00:00Z"... 

Comment by yuki.moritani [ 14/May/22 ]

I'm not sure it works to you, but if you run the code via notebook, could you try adding these lines before running the code?

 

 

from astropy.utils.data import clear_download_cache
clear_download_cache()
from astropy.utils import iers
# iers.IERS_A_URL
iers.Conf.iers_auto_url.set('https://datacenter.iers.org/data/9/finals2000A.all')
from astroplan import download_IERS_A
download_IERS_A()

 

Comment by Martin Reinecke [ 14/May/22 ]

OK, then this really looks like the tables in astropy need some kind of update. I was worried that the time string was getting scrambled somehow on my side, but this doesn't seem to be the case.

Comment by Kiyoto Yabe [ 14/May/22 ]

Thank you, yuki.moritani . We already have actually changed the url to different sites and tested. The suggested lines actually didn't work unfortunately. Yes, it looks like the astropy problem.

Comment by monodera [ 14/May/22 ]

There was some discussion on astropy and they introduced a config parameter iers_degraded_accuracy.

https://docs.astropy.org/en/latest/utils/iers.html#configuration-parameters
https://github.com/astropy/astropy/pull/13052

I tried to install the latest version (i.e., the master branch) of astropy and set 

from astropy.utils import iers
iers.conf.iers_degraded_accuracy = "warn"

at the top of the script. Then, it looks working by switching to the IERS-B data. I'm not sure this is the right way, but I hope it is a reasonable workaround.

Comment by Martin Reinecke [ 14/May/22 ]

Concerning the question how I can correct RA/Dec in a way that is consistent with pfs_utils: I think the relevant code snippet is https://github.com/Subaru-PFS/pfs_utils/blob/tickets/INSTRM-1582/python/pfs/utils/coordinates/DistortionCoefficients.py#L608-L633

I think I can copy this into my code, but if there is a preference to have this as a standalone helper function in pfs_utils, I'm happy to provide a PR for that!

What do you think, yuki.moritani, rhl?

Comment by yuki.moritani [ 14/May/22 ]

If needed, calculated ra-dec can be added as return of that routine.. so that you can call it.

Comment by Kiyoto Yabe [ 15/May/22 ]

Adding

from astropy.utils import iers
iers.conf.auto_download = True

works for me now.

I don't know why because the default value is `True`.

Comment by Kiyoto Yabe [ 15/May/22 ]

With this version, it seems that the relative geometry between AGs and cobras is bit strange. Attached is an example FoV map (on sky) with coordinates of assigned guide objects and science targets. 

(detailed command with Onodera-san's script is like this)

python $REPO/subaru_fiber_allocation.py --design_dir "design/prep/yamashita" --cobra_coach_dir "cobracoach"  --ra 268.70 --dec 20.27  --pa -90.0  --fluxstd_min_prob_f_star 0.0  --n_fluxstd 500  --guidestar_mag_max 20.0  --observation_time 2022-05-19T10:30:00Z  --conf config.toml --pfs_instdata_dir  /work/pfs/commissioning/fiber_allocation/pfs_instdata

Changing PA doesn't change the relative geometry. I think this didn't happen at least in Nov., and I still don't know when this was changed. Or perhaps I did something stupid...

Comment by yuki.moritani [ 15/May/22 ]

Let me check...

Comment by Kiyoto Yabe [ 16/May/22 ]

Thank you yuki.moritani for your suggestion offline.

OK, monodera : Is the script pass the same PA to both netflow part and guide star selection part?

According to the code, it looks yes.

Then, Martin Reinecke : Are the netflow (or nf.Telescope) and the guide star part using the same PA?

The guide positions should be around sides of the hexagon (not vertices). See the movie in the middle of the official website (https://pfs.ipmu.jp/instrumentation.html). Sorry but I cannot quickly show a better photo.

Comment by Martin Reinecke [ 16/May/22 ]

If needed, calculated ra-dec can be added as return of that routine.. so that you can call it.

Do you mean by introducing a new transform mode, e.g. "sky_to_corrected_sky"? That would be a great addition!

Comment by Martin Reinecke [ 16/May/22 ]

Then, Martin Reinecke : Are the netflow (or nf.Telescope) and the guide star part using the same PA?

The "guidecam_geometry" function is independent of the PA (as far as I understand), since it works entirely with local PFI coordinates. All the remaining transforms are probably taking part inside the "subaru_fiber_allocation.py" script. If you can point me to the location of this script, I'm happy to look more closely. (I assume it has evolved from "subaru_fiber_allocation_2021nov.py", but I don't know where it is located.)

Comment by monodera [ 16/May/22 ]

Hi, the script for this commissioning run can be found here (commissioning_2022may branch of ets_target_database).

https://github.com/Subaru-PFS/ets_target_database/tree/commissioning_2022may/examples/commissioning_2022may

Comment by monodera [ 18/May/22 ]

I think I was able to fix the issue following the advice by Yabe-san and Martin. I found a bug in my script. Thanks a lot.

I attached some example pfsDesigns by using the following command.

python ./subaru_fiber_allocation.py \
    --conf ../../../database_configs/config_pfsa-db01-gb_commissioning_2022may.toml \
    --ra 161.72486 \
    --dec 28.53286 \
    --pa -90.0 \
    --cobra_coach_dir coach \
    --design_dir=design \
    --n_fluxstd 0 \
    --fluxstd_min_prob_f_star 0.0 \
    --guidestar_mag_max 20.0 \
    --observation_time 2022-05-19T10:30:00Z \
    --exptime 900 \
    --raster_scan \
    --raster_mag_max 20

Then by setting --sm option to specify which spectrographs to be used, one can limit cobras to be allocated to objects.

pfsDesign-0x78e3b3034e015cf0_sm1+2+3+4.pdf : All spectrographs
pfsDesign-0x413bcf208d3daa22_sm1+3.pdf : SM1 and SM3
pfsDesign-0x6b7813a6e0eb4ef9_sm1.pdf : SM1

In the figures, a large red circle indicates the location of a calibration star provided by Yamashita-san and you can see it is allocated to SM1. Other allocated objects are stars used for the raster scan test.

Comment by monodera [ 18/May/22 ]

AG star locations look also okay.

Comment by yuki.moritani [ 10/Jun/22 ]

It was tested and used by a few colleagues last run. As it's been merged to master, I close this ticket.

Generated at Sat Feb 10 16:37:08 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.