[OBSPROC-55] Input format for filter and flux information for the target uploader

Relates — Wed, 18 Oct 2023 18:49:44 +0000

I'm wondering how we ask users to put flux and filter information. I can think of two possibilities.

Assumptions

The unit of fluxes is nJy. (endorsed by Tanaka-san)
The fluxes are the (user-defined) total flux. It is equivalent to the PSF flux for point sources.
Filter names must be in our pre-defined list which is closely related to the data reduction pipeline.

1. Follow the schema of the `targetdb`

The targetdb has columns like filter_{grizyj} and psf_flux_{grizyj} and psf_flux_error_{grizyj}. We can add total_flux_* columns in the targetdb and alias to flux_* in the input list. Then, we can ask users to use them as columns of the input list. The flux-related columns looks like the following.

obj_id,filter_r,flux_r,filter_j,flux_j,flux_error_j
1,rp_gaia,1000000,,,
2,bp_gaia,100000,,,
3,i2_hsc,10000,j_mko,20000,200

Pros

Flexible: Each object can have different filters for the same column name.

Cons

Users may need to spend more time to create a list.
Correspondence between filter names and column subscripts can be inconsistent. For example, one can bp_gaia in filter_r (the above example), while it's entirely valid and has no problem downstream, I suppose.

2. Use filter names in the column name

The other possibility is that users use the pre-defined filter name in the column header in the input list. The example list in the case 1 looks like the following in this case.

obj_id,rp_gaia,bp_gaia,i2_hsc,j_mko,j_mko_error
1,1000000,,,,
2,,100000,,,
3,,,10000,20000,200

Pros

Looks more straightforward for users?
Simpler when the list is generated from a single parent catalog (e.g., HSC-SSP)

Cons

We need to define the correspondence between filtername and columns in targetdb.
It's not possible to list more than one filter for an object. For example, r2_hsc and r_ps1 are regarded as _r filters, and cannot co-exist if we set filter categories.

Perhaps, validation is equivalently simple or complex and can be implemented anyway. Any other ideas are also welcome. We just need to decide to go forward.

PFS-JIRA

[OBSPROC-55] Input format for filter and flux information for the target uploader

Assumptions

1. Follow the schema of the targetdb

Pros

Cons

2. Use filter names in the column name

Pros

Cons

1. Follow the schema of the `targetdb`