Skip to content

Implementation suggestions for HyperParameters class

I have implementation suggestions for HyperParameters class.

The HyperParameters.__add__ works in a way where both of the operands are being copied and a new HyperParameters object is returned.

However, for in place addition as in param1 += param2, the object.__iadd__ method should be used. In this case, only param2 will need to be deep copied and the param1 can be updated accordingly. This should be a much elegant and more memory saving operation.

I don't know if this is a good thing or not, but since deep copying can be expensive (what if param2 has network weight already loaded?), maybe we can add a toggle variable in HyperParameters like HyperParameters.__deepcopy. For example, param1 can check param2.deepcopy (maybe @property can be used to access HyperParameters.__deepcopy) during the __iadd__ operation to determine weather the variables in param2 should be deep copied or not.

By default, HyperParameters.__deepcopy will be set to True, but power users that know what they are doing can toggle it maybe by calling HyperParameters.disable_deepcopy.