Return read-only collections from @cached methods

[squahtx]

It's important that collections returned from @cached methods are not
modified, otherwise future retrievals from the cache will return the
modified collection.

This applies to the return values from @cached methods and the values
inside the dictionaries returned by @cachedList methods. It's not
necessary for the dictionaries returned by @cachedList methods
themselves to be read-only.

Signed-off-by: Sean Quah [email protected]

[squahtx]

This goes straight into the SyncResult and is pretty much only going to be JSON-serialized from here on.
Having to take a copy to make mypy happy is unfortunate.

[DMRobertson]

At least it's a shallow copy 🤷

[clokep]

I'd be curious of the error here? Is something in our code unhappy or is it a type hint of like json.dump?

I think it'd be OK to add a cast with a comment explaining it is for performance.

[squahtx]

It's a type hint problem. Only Lists and Tuples are JSON-serializable, except they're mutable types. I would have updated the definition of SyncResult otherwise.

[DMRobertson]

I haven't looked through the full changes (sorry), but I'd love to see us land some of this: particularly bugfixes or things that don't incur performance costs. (Though it's Python: if you're worried about performance costs, it could be argued you're doing something wrong.)

One thing I'm not clear on is enforcement. Is there some way we can annotate @cached or some mypy plugin witchcraft we can do to which will force us to only cache functions with immutable return values? (Possibly we could use Hashable as an approximation of an Immutable type?)

[DMRobertson]

Is there some way we can annotate @cached or some mypy plugin witchcraft we can do to which will force us to only cache functions with immutable return values? (Possibly we could use Hashable as an approximation of an Immutable type?)

I indulged myself. #14911 is a mypy plugin which hopefully detects a bunch of these.

[DMRobertson]

I've tried to merge in develop to resolve the merge conflicts. (I'd like to see this land).

(And I also want to see what #14911 makes of it)

[DMRobertson]

I've tried to merge in develop to resolve the merge conflicts. (I'd like to see this land).

(And I also want to see what #14911 makes of it)

Against develop: 181 errors
With this PR: 72 errors (some of which may be my hacky plugin).

[DMRobertson]

Huge thanks for this Sean. I think this is a great change and am looking forward to being shot at in the foot less. Just a few things to check:

[DMRobertson]

LGTM!

[squahtx]

I had to fix some merge conflicts.