Commit 41fcea09 authored by Mitar's avatar Mitar

Merge branch 'devel' into no-enumerations

parents 84ce005e 71bf9013
Pipeline #129347753 passed with stage
in 4 minutes and 7 seconds
......@@ -5,6 +5,10 @@
[#132](https://gitlab.com/datadrivendiscovery/ta3ta2-api/-/issues/132)
[#133](https://gitlab.com/datadrivendiscovery/ta3ta2-api/-/issues/133)
[!167](https://gitlab.com/datadrivendiscovery/ta3ta2-api/-/merge_requests/167)
* Added `use_default_values_for_free_hyperparams` flag to `SearchSolutionsRequest`
to precisely control the behavior when a search template contains no placeholders
but free hyper-parameters.
[!168](https://gitlab.com/datadrivendiscovery/ta3ta2-api/-/merge_requests/168)
* Fixed URL validation in Python utils when an allowed directory path
contains a trailing `/`.
[!165](https://gitlab.com/datadrivendiscovery/ta3ta2-api/-/merge_requests/165)
......
......@@ -175,7 +175,7 @@ primitive is used.
Relaxation: Individual systems can relax those restrictions. For example, they might allow
a placeholder step to have postprocessing primitive steps after it. In this case postprocessing
primitives can only transform predictions from a placeholder step into transformed predictions.
Or individual systems might allow primitve steps to have free hyper-parameters a TA2 system
Or individual systems might allow primitive steps to have free hyper-parameters a TA2 system
should tune.
### Fully specified pipelines
......@@ -194,10 +194,11 @@ to convert input value to closest container type value, e.g., GRPC `RAW` list va
be converted to `d3m.container.List` with generated metadata, CSV file read as Pandas
DataFrame should be converted to `d3m.container.DataFrame` with generated metadata.
Relaxation: Individual systems can support also pipelines with all primitives specified,
Individual systems can also support pipelines with all primitives specified,
but with free (available for tuning) hyper-parameters. In this case, TA2 will only tune
hyper-parameters and resulting pipelines will have the same structure as given pipeline,
but hyper-parameter configuration will differ.
but hyper-parameter configuration will differ. If such potential behavior of a system is
not desired, a `use_default_values_for_free_hyperparams` flag can be set to true.
## Values
......
......@@ -135,8 +135,9 @@ message SearchSolutionsRequest {
// regular solution search is done. If template consists only of one placeholder step,
// then a regular solution search is done to replace that step. If there is no placeholder
// step, but template describes a full pipeline with free hyper-parameters, then this
// call becomes a hyper-paramater tuning call over free hyper-parameters and found solutions
// share the same pipeline, but different hyper-parameter configurations. If there is no
// call becomes a hyper-parameter tuning call over free hyper-parameters and found solutions
// share the same pipeline, but different hyper-parameter configurations (unless
// "use_default_values_for_free_hyperparams" is set to true). If there is no
// placeholder step and all hyper-parameters are fixed as part of the pipeline, then this
// call only checks the given template and returns the solution with same pipeline back, to
// be executed. This allows fixed computations to be done on data, for example, pipeline can
......@@ -148,7 +149,8 @@ message SearchSolutionsRequest {
// systems. Minimum (baseline) requirements are that all TA2's support:
// - Omitted templates,
// - Partial pipelines with one placeholder step, at the last step in the pipeline template,
// - Fully specified pipelines without free hyper-parameters.
// - Fully specified pipelines without free hyper-parameters, or with free hyper-parameters
// and "use_default_values_for_free_hyperparams" set to true.
// See the Pipeline section of the README for more details, in particular the discussion of
// common requirements and possible relaxations.
PipelineDescription template = 7;
......@@ -186,6 +188,17 @@ message SearchSolutionsRequest {
// Random seed used to initiate search. Note, not setting this value means
// always using seed of 0.
int32 random_seed = 11;
// If set to true, during search TA2 is suggested to not do hyper-parameter tuning
// of free hyper-parameters but use their default values. It might be that TA2
// operates searches for both the pipeline and hyper-parameter values at the same
// time and flag cannot be easily respected, in that case this flag can be ignored by TA2.
// When template is provided and contains free hyper-parameters but no placeholders,
// then setting this flag to true means that for all free hyper-parameters in the
// template default values should be used. This allows TA3 to not have to fix
// values for all free hyper-parameters in the template itself, to obtain a
// fully specified pipelines without free hyper-parameters.
// Generally applies only to tuning hyper-parameters.
bool use_default_values_for_free_hyperparams = 12;
}
// Call returns immediately with the ID. Use "GetFoundSolutions" call to get results.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment