Hyper-parameter type for set of hyper-parameters

It seems there is a need of defining a (sub)set of hyper-parameters, especially because we do not yet have dependencies between hyper-parameters. For example, KNeighborsClassifier has a metric and metric_params hyper-parameters. While in the future we might expose those as primitive inputs, we could for now also see it as a enumeration of string names for those directly supported by sklearn. And each name has then a set of configuration hyper-parameters. It could look like:

metric = hyperparams.Union(OrderedDict(
  metric1=hyperparams.Set(OrderedDict(
    param1_for_metric1=...,
    param2_for_metric1=...,
  )),
  metric2=hyperparams.Set(OrderedDict(
    param1_for_metric2=...,
    param2_for_metric2=...,
    param3_for_metric2=...,
  )),
))

Maybe we could reuse Hyperparams as a set and allow it to be Hyperparameter?

The question is also how does the primitive then know which of Union options were chosen? Just from what is available in the set chosen?

Edited by Mitar