sardana config: serializing ambigious attribute values
I think we have a tricky issue with the (de)serialization of attribute values, that often contain various configuration stuff like nested arrays, stored as strings. Currently we use JSON serialisation to turn values from YAML into nice strings, and this probably works for a fairly large class of cases. But since generally there's no rules for how this should be done, the controller may expect it to be formatted in any way. We just found the following case that currently breaks (using the fix-config-dump-issues
branch):
The attribute Configuration is of type DevString and has the value [[-1000, -3.5, -3.4], [-1, 0, 1]]
.
config dump
turns this into the following YAML:
Calibration:
- - -1000
- -3.5
- -3.4
- - -1
- 0
- 1
Pretty ugly, but seems correct in principle. However, turning this back into dsconfig JSON (using python3 -m sardana.config.yaml2dsconfig
) yields this:
"Calibration": {
"__value": [
"[-1000, -3.5, -3.4]",
"[-1, 0, 1]"
]
},
This is clearly incorrect, it should be the string "[[-1000, -3.5, -3.4], [-1, 0, 1]]"
and not two strings. This happens because of the stringify
function trying to be clever. Since we don't know what type the attribute actually has, we just try to guess what it might be, and if it's a list in the YAML we naively think it might be an array attribute and handle it like that. But this is clearly not going to work in general.
So what to do? If we knew what type the attribute has, we could make better decisions here, but it's not available from the DB. So far we haven't strictly required access to the controller code for load/dump, only for the "code check" step. But since that step should be almost mandatory anyway (I think) we could do it here too. We already have the needed code, and would just get the type for each attribute both when dumping and loading. But we'd still have the issue of not knowing how to encode strings. The simplest solution would be to also just require strings to be strings in YAML, e.g.
Configuration: "[[-1000, -3.5, -3.4], [-1, 0, 1]]"
Not super pretty, but I actually think most people would prefer it to the weird nested list above. Now, if such attributes would be specified as devencoded, we could of course really know what codec to use and (de)serialize properly! Not sure if that is supported by sardana though, and we would still have to deal with all the existing controllers that don't do this. Perhaps storing devstring attributes as strings in YAML is the only safe route anyway...
This could become a big issue, I don't really have a feeling for what weird configuration formats might be in use out there, probably we should try to be cautious here or we could have a huge source of bugs on our hands. I think it would be good to survey this a bit on our respective facilities. I.e. what types/encodings of attributes are actually in use on controllers and elements?
I guess this problem also applies to properties, by the way.