[INSTRM-1091] Add butler persistence to/from opDb

Fri, 22 Apr 2022 03:12:36 +0000

The ICS butler in pfs_utils currently persists to files, generally either YAML or FITS. Per many recent discussions, some objects need to be persisted to opDb instead. Support that for things like:

cobra = self.butler.get('cobraGeometry', idDict)
self.butler.put(cobra, 'cobraGeometry', idDict)

The filesystem persistence mechanism generally did not address versioning, leaving that up to external tools and procedures (e.g. someone calling 'git commit' for updated files in pfs_instdata).

Likewise with the opDb. If the schema supports persistence, e.g. with an update time column, then we get versioning. Else not.

The existing filesystem map for the above example is:

configMap['cobraGeometry'] = dict(template="pfi/cobras/{moduleName}/{cobraInModule}/geometry.yaml",
     loaderModule='ics.cobraCharmer.cobra')

with a magic load(path) in the loaderModule, and a .dump(path) in the returned class.

I do not have confidence that there will always be a one-to-one object-to-table mapping, so am not sure whether naming a table in the butler is useful. So I guess I'd make that:

configMap['cobraGeometry'] = dict(opDb=True,
                                  loaderModule='ics.cobraCharmer.cobra')

and add loadOpDb(idDict) and .dumpOpDb(idDict) function & method. Allow useful info to be stuffed in idDict.

Use the spt_operational_database library routines to wrap opDb access, so I do not need to think about database implementation details.

PFS-JIRA

[INSTRM-1091] Add butler persistence to/from opDb