[INSTRM-1640] FiberID sometimes takes too long Created: 20/Jun/22 Updated: 02/Sep/22 Resolved: 21/Jul/22 |
|
| Status: | Done |
| Project: | Instrument control development |
| Component/s: | ics_mcsActor |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Normal |
| Reporter: | cloomis | Assignee: | karr |
| Resolution: | Done | Votes: | 0 |
| Labels: | EngRun | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Sprint: | preEngRun07Sep |
| Description |
|
Fiber ID matching took ~35s. This is with the non easyId algorithm, and happened several times. 2022-06-19 23:44:45.204Z cmds 20 CommandLink.py:121 > 2 280 i text="loaded 2394 targets from DB" 2022-06-19 23:44:45.205Z cmds 20 CommandLink.py:121 > 2 280 i text="Starting Fiber ID" 2022-06-19 23:45:19.401Z cmds 20 CommandLink.py:121 > 2 280 i text="Fiber ID finished" 2022-06-19 23:45:19.689Z cmds 20 CommandLink.py:121 > 2 280 i text="wrote matched cobras to database" 2022-06-19 23:45:19.689Z cmds 20 CommandLink.py:121 > 2 280 i frameId=7739802; filename=/data/raw/2022-06-19/mcs/PFSC07739802.fits 2022-06-19 23:45:19.690Z cmds 20 CommandLink.py:121 > 2 280 : exposureState=done |
| Comments |
| Comment by karr [ 20/Jun/22 ] |
|
I'll take a look at that. It definitely shouldn't be taking that long! |
| Comment by karr [ 20/Jun/22 ] |
|
Do you have examples of frameIds where this happens? |
| Comment by cloomis [ 20/Jun/22 ] |
|
(In the log segment): 7739802 |
| Comment by karr [ 20/Jun/22 ] |
|
I've figured out what's causing the problem - in some cases when there are a lot of ambiguous fibre identifications, the determination of the distance from target is being inefficiently. I have the idea of what will fix it, but need to work out the complex book-keeping to go with tracking all the assignments through the loop,. |
| Comment by karr [ 20/Jun/22 ] |
|
Spoke too soon. The distance calculation is lightning fast (using cdist); the bottleneck was the use of argsort to find the global closest matches between unassigned fibres and unassigned cobras. Have replaced argsort with the use of argpartition, as it's only the closest points that are of interest; this is a significant improvement. We still probably want it faster, but I need to think about how to do so. I've pushed an update, but I want to run it though some more tests to be really sure that I haven't broken anything.
|
| Comment by karr [ 08/Jul/22 ] |
|
There are a couple of modifications that will speed things up in extreme cases, like the dot crossing routines, mostly via complicated book-keeping so we don't need to recompute distances between points and targets. In addition, for regular convergence runs, once the cobra_target table is properly populated, we can save time by doing fast matching at the beginning of the sequence for no longer moving cobras, the way we match the fiducial fibres.
|
| Comment by karr [ 14/Jul/22 ] |
|
Implemented the first set of changes.
|
| Comment by hassan [ 16/Jul/22 ] |
|
karr will close this ticket based on the first set of changes, and file a new ticket for the work related to the cobra_target table. |
| Comment by karr [ 21/Jul/22 ] |
|
Merged to master.x |