[OBSPROC-54] Assignment of input_catalog_id Created: 03/Oct/23  Updated: 14/Dec/23

Status: Open
Project: PFS observation processing/procedure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: monodera Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: targetDB
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: targetDB

 Description   

Currently, the primary key of the input_catalog table, input_catalog_id , is hard-coded like

  catId | description
 -------|--------------------------------------------------
      0 | Simulated catalog
      1 | Gaia Data Release 1
      2 | Gaia Data Release 2
      3 | Gaia Early Data Release 3
      4 | Gaia Data Release 3
      5 | HSC-SSP Public Data Release 1 (Wide)
      6 | HSC-SSP Public Data Release 1 (Deep+UltraDeep)
      7 | HSC-SSP Public Data Release 2 (Wide)
      8 | HSC-SSP Public Data Release 2 (Deep+UltraDeep)
      9 | HSC-SSP Public Data Release 3 (Wide)
     10 | HSC-SSP Public Data Release 3 (Deep+UltraDeep)
     11 | HSC-SSP Public Data Release 4 (Wide)
     12 | HSC-SSP Public Data Release 4 (Deep+UltraDeep)
   1001 | Sky positions from S21A HSC-SSP (Wide)
   1002 | Sky positions from PS1
   1003 | Sky positions from Gaia
   1004 | Sky positions for regions without PS1 data
  90001 | Messier 15 stars for the engineering observation
  90002 | NGC1904 for the engineering run in November 2022
  90003 | NGC2419 for the engineering run in November 2022
  90004 | NGC5272 for the engineering run in February 2023
  90005 | NGC5904 for the engineering run in February 2023
  90006 | NGC7078 for the engineering run in July 2023
  90007 | NGC7089 for the engineering run in July 2023
  90008 | NGC7099 for the engineering run in July 2023

This implementation would work when the number of entries is small. For the openuse operation, we need to think about how to handle input catalogs provided by the user. It is highly likely that users want to use catalogs from their own observations, reductions, etc., not listed above or not covered by the large surveys with their own source ID scheme.

My proposal here is the following.

  • Assign an entry in the input_catalog table for each submission via the target uploader. Therefore, if one program submits two lists, two input_catalog_id are assigned.
    *Make the input_catalog_id auto-increment primary key with the starting ID of, say, 100001.
  • (optional) If we need to add catalogs from major surveys not necessarily associated with openuse programs, we can force to use younger numbers.
  • (optional) We may allow users to indicate parent catalogs if they are in our pre-defined list and host this information in a column in the target table (something like parent_catalog_id?).

Any comments?

 Comments   
Comment by monodera [ 03/Oct/23 ]

I'm wondering if OCS team has any comments especially from the point of view of progress tracking.eric Russell Kackley

 

 

Comment by monodera [ 14/Dec/23 ]

Comment from Tanaka-san

 

  • How are we going to define catId by the user? We certainly do not want individual PI to define a random catId because (catId, objectId) has to be unique. I think it makes sense to assign (tentative) catId at the uploader to be sure that different PIs use different IDs. If we want to adopt different numbering schemes for SSP, intensive, normal, ToO, and fillers, then we should reassign the Ids when we accept proposals. For instance, "catId = %2d%1d%2d" % (semesterId, proposalCategory, sequentialNumber) ??? Note that we use catId in file names and it should be less than 5 digits (well, we can increase it if needed and before it is too late).

 

Current datamodel requests that catId is 5digit integer, but we can change it if needed.

Generated at Sat Feb 10 16:44:47 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.