[OBSPROC-30] Review the pfsDesign generation script in 2022Sep run and plan for 2022Nov run

Blocks — Tue, 29 Nov 2022 18:37:49 +0000

What we noticed during the 2022Sep run
- general
  - The designed OBSTIME is not persisted anywhere. I remember RHL liked to save such information outside the datamodel (maybe queue or processing-related DB?).
    - --> The information is stored in log file so far
- script
  - Currently, the scripts are located on `ets_target_database/examples/commissioning_2022sep` for the historical reason, we should move them to a canonical place (see ~~OBSPROC-25~~).
    - --> The scripts are in `ets_pointing` repository now
  - Also, maybe we should fit the `EUPS` mechanism so that we can switch to different versions, but need to check if this is possible.
  - FiberStatus is not propagated to pfsDesign (see ~~INSTRM-1748~~)
    - --> Fixed
- fiber allocation
  - Do we need to control the calibrators (sky and fluxstds) not to be closed to bright fibers?
  - The assigned flux standard stars are too faint for the further study of the flux calibration, so next time assign as bright flux standards as possible
    - --> It seems that the flux standards are relatively faint in the NGC1980 field unfortunately. There are bright stars assigned in different fields.
- targetDB
  - targets
    - `target` table has columns for epoch and proper motion but pfsDesign datamodel not (see ~~DAMD-137~~).
  - sky objects
    - The populated `sky` table was huge so it took very long to make a query for sky fibers. During the run, the coordinate of objects was indexed with `q3c_radial_query` mechanism and query speed was significantly improved. We should think about more in future (see ~~OBSPROC-29~~).
      - --> We will keep using `q3c` for a while
    - On the other hand, in some regions, the number density of the sky objects was very high so it took long time for the netflow and collision avoidance optimization. In the end of the run, we randomly picked up the fetched sky targets and reduced the number so that the processing time was reasonable short.
      - --> We will use the same mechanism for a while
    - For `obj_id`, `string` and `int64` were mixed
      - --> Fixed?
    - We couldn't utilized the file of mapping between tract information and target coordinates
    - We should think about more on how to make sky targets in a field with diffuse feature and very high target density (such as M31).
- gaiaDB
  - We moved to Gaia DR3 database cloned in the summit/Hilo. There was no change in queries other than the table name from `gaia` to `gaia3`.
Plan for the next run
- Fix scripts for above issues before the next run

PFS-JIRA

[OBSPROC-30] Review the pfsDesign generation script in 2022Sep run and plan for 2022Nov run