[INSTRM-1800] Characterize/fix reqframes response

Fri, 2 Feb 2024 01:09:21 +0000

The reqframes service sometimes takes quite a long time to respond. In the latest case about 100s:

2022-11-21 01:56:20,661 | I | PFS.py:654 (reqframes) | reqframes num=100 type='A'
[ ... ]
2022-11-21 01:57:59,327 | I | PFS.py:658 (reqframes) | framelist:

Out of the 44230 reqframes calls made in 2021-2, here are the times for all calls which took more than 1 second. We currently have a 10s timeout, so fourteen real failures for us.

2021-01-12 14:18:25.453000 2
2021-01-14 12:45:13.123000 2
2021-01-17 10:30:41.375000 2
2021-01-19 12:39:47.543000 2
2021-01-22 06:38:42.979000 3
2021-01-24 22:11:03.833000 5
2021-01-30 05:31:01.442000 3
2021-02-02 23:38:03.112000 4
2021-02-06 02:36:59.753000 4
2021-02-22 00:58:19.438000 5
2021-02-25 15:09:51.480000 2
2021-03-31 12:34:28.069000 3
2021-05-04 15:53:02.737000 2
2021-05-06 20:06:15.625000 2
2021-05-07 15:59:53.094000 2
2021-05-08 00:09:19.605000 2
2021-05-10 09:17:46.669000 2
2021-05-10 21:48:53.330000 3
2021-05-13 16:22:39.357000 5
2021-05-15 18:08:46.252000 4
2021-05-17 12:21:06.005000 5
2021-05-20 12:59:46.067000 7
2021-05-21 19:20:02.311000 7
2021-05-25 13:38:35.847000 9
2021-05-31 05:23:14.546000 11
2021-06-03 00:07:58.940000 15
2021-06-07 11:04:03.625000 15
2021-06-24 01:58:23.225000 26
2021-09-12 22:30:38.936000 2
2021-09-15 02:28:04.480000 5
2021-10-22 01:17:29.340000 2
2021-11-20 05:27:04.965000 2
2021-11-21 05:44:35.302000 3
2021-11-21 16:05:50.479000 14
2021-11-21 17:12:52.306000 3
2021-11-21 17:17:16.721000 2
2021-11-21 17:18:32.524000 2
2021-11-21 17:33:25.710000 2
2021-11-21 18:28:36.297000 4
2021-11-21 19:24:34.245000 3
2021-11-21 21:38:29.590000 90
2021-11-22 03:08:22.385000 3
2022-04-30 22:21:00.018000 3
2022-05-06 19:47:37.766000 17
2022-05-11 12:00:19.542000 5
2022-05-11 12:00:27.926000 5
2022-05-11 12:00:35.300000 5
2022-05-11 12:00:40.870000 3
2022-05-11 12:10:12.553000 2
2022-05-11 14:26:41.336000 2
2022-05-11 20:21:44.130000 3
2022-05-11 20:23:24.809000 2
2022-05-11 20:24:34.191000 2
2022-05-11 20:28:02.424000 2
2022-05-11 20:31:45.505000 2
2022-05-11 20:34:15.027000 3
2022-05-11 20:49:41.250000 3
2022-05-11 20:53:52.303000 2
2022-05-11 20:56:53.594000 127
2022-05-12 07:39:15.808000 26
2022-05-12 10:13:46.341000 2
2022-05-12 10:21:57.150000 7
2022-05-12 11:41:35.705000 15
2022-05-16 19:14:26.932000 10
2022-05-16 19:35:25.384000 2
2022-05-16 19:56:23.078000 2
2022-05-16 20:17:20.380000 2
2022-05-17 16:08:31.585000 2
2022-05-17 16:34:17.428000 2
2022-05-17 16:53:57.678000 2
2022-06-09 09:51:07.808000 11
2022-06-14 10:36:39.119000 8
2022-06-19 20:13:01.616000 2
2022-06-22 07:26:24.785000 3
2022-09-28 18:36:28.480000 3
2022-11-07 22:06:21.761000 3
2022-11-13 23:57:46.388000 7
2022-11-14 03:15:06.610000 2
2022-11-16 19:12:48.739000 38
2022-11-17 07:57:02.254000 7
2022-11-18 20:16:55.378000 2
2022-11-21 01:57:59.327000 98

PFS-JIRA

[INSTRM-1800] Characterize/fix reqframes response