Skip to content

normalization by input size

Normalize entropy relative to the length of the input data. For example, when the domain cardinality of the input data is greater than the length input data, it is impossible to achieve maximum entropy as currently calculated by the code, even if the input is uniformly distributed. Some normalization is needed to properly compare two input examples of different lengths, where their lengths are < the domain cardinality.