[OBSPROC-28] duplication of objId in pfsDesign.guideStars Created: 17/Sep/22 Updated: 18/Sep/22 Resolved: 18/Sep/22 |
|
| Status: | Done |
| Project: | PFS observation processing/procedure |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Normal |
| Reporter: | Kiyoto Yabe | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
There seems to be duplication of objId in `pfsDesign.guideStars` of `/data/pfsDesign/pfsDesign-0x74f2c06289c9823d.fits`. The objId is 6034944. This causes `UniqueViolation` error when ingesting to `pfs_design_agc` table. Investigate that and fix if it is real. |
| Comments |
| Comment by monodera [ 17/Sep/22 ] |
|
In the datamodel code, objId for guide stars are defined as int32, while it is documented as int64 in datamodel.txt. I think this discrepancy causes an unintended rounding of int64 to int32 when reading a pfsDesign file. |
| Comment by hassan [ 17/Sep/22 ] |
|
You are correct monodera that there is a bug in the datamodel code. Fix in ticket branch of from pfs.datamodel.pfsConfig import PfsDesign a = PfsDesign.read(0x74f2c06289c9823d, dirName='.') objId = a.guideStars.objId assert(len(set(objId)) == len(objId)) print('ObjectIds are unique') |
| Comment by hassan [ 18/Sep/22 ] |
|
Fix for |
| Comment by Kiyoto Yabe [ 18/Sep/22 ] |
|
Thank you! I confirmed that the problem resolved. arnaud.lefur I think now you can ingest guidestars info into pfs_design_agc table using the latest datamodel package. |
| Comment by arnaud.lefur [ 18/Sep/22 ] |
|
Thanks ! Yes I just tested and ingestPfsDesign works fine now. IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "pfs_config_agc_pkey"
DETAIL: Key (pfs_design_id, visit0, guide_star_id)=(8427009382214631997, 78607, 6034944) already exists.
[SQL: INSERT INTO pfs_config_agc (pfs_design_id, visit0, guide_star_id, agc_camera_id, agc_final_x_pix, agc_final_y_pix) VALUES (%(pfs_design_id)s, %(visit0)s, %(guide_star_id)s, %(agc_camera_id)s, %(agc_final_x_pix)s, %(agc_final_y_pix)s)]
[parameters: ({'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 6673024, 'agc_camera_id': 0, 'agc_final_x_pix': 1017.9483642578125, 'agc_final_y_pix': 827.9166259765625}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 6034944, 'agc_camera_id': 0, 'agc_final_x_pix': 946.2750244140625, 'agc_final_y_pix': 902.5942993164062}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 6033152, 'agc_camera_id': 0, 'agc_final_x_pix': 896.0719604492188, 'agc_final_y_pix': 912.3250122070312}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 6031872, 'agc_camera_id': 0, 'agc_final_x_pix': 810.6386108398438, 'agc_final_y_pix': 923.0083618164062}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 6663936, 'agc_camera_id': 0, 'agc_final_x_pix': 960.7747192382812, 'agc_final_y_pix': 838.6574096679688}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 21225472, 'agc_camera_id': 0, 'agc_final_x_pix': 962.1170654296875, 'agc_final_y_pix': 819.9133911132812}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 21219712, 'agc_camera_id': 0, 'agc_final_x_pix': 840.3349609375, 'agc_final_y_pix': 862.7570190429688}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 21228416, 'agc_camera_id': 0, 'agc_final_x_pix': 881.2999267578125, 'agc_final_y_pix': 726.2146606445312} ... displaying 10 of 556 total bound parameter sets ... {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 21866112, 'agc_camera_id': 5, 'agc_final_x_pix': 50.762168884277344, 'agc_final_y_pix': 1008.1949462890625}, {'pfs_design_id': 8427009382214631997, 'visit0': 78607, 'guide_star_id': 9773824, 'agc_camera_id': 5, 'agc_final_x_pix': 35.872276306152344, 'agc_final_y_pix': 827.4269409179688})]
If you want to reproduce the problem, I've put the pfsConfig file I just generated there : Thanks. |
| Comment by hassan [ 18/Sep/22 ] |
|
That pfsConfig file you provided does contain a single duplicate guide_star_id value of 6034944. But how was it generated? Was it generated using the fix to datamodel in |
| Comment by arnaud.lefur [ 18/Sep/22 ] |
|
So, as hassan pointed out, the pfsConfig file needed to be regenerated. |