Commit a0e423e4 authored by D3M CI's avatar D3M CI

Generated by CI.

Source commmit: 1e8f0c90

[skip ci]
parent 1214abaa
......@@ -14,8 +14,8 @@ about GRPC.
## API Structure
TA3-TA2 API calls are defined in the *core* GRPC service which can be found in [`core.proto`](./core.proto) file
and all TA3 and TA2 sytems are expected to implement it and support it. Optional services can be
defined as well in other `.proto` files.
and all TA3 and TA2 systems are expected to implement it and support it. Other `.proto` files provide definitions
of additional standard messages.
Useful utilities for working with the TA3-TA2 API in Python are available in the included [ta3ta2_api](https://gitlab.com/datadrivendiscovery/ta3ta2-api/tree/dist-python) package.
......@@ -136,11 +136,6 @@ for partially specified pipelines; the problem description for a partially
specified pipeline should describe the data at the beginning of the pipeline,
not the end of the specified portion.
Examples of "relaxations" of the common requirements are included. We expect
that some TA2 systems will be able to work with those relaxed requirements,
and TA3s can use those if available, but it is not expected that every TA2
will. TA3s should be able to function within the restrictions as stated below.
### Pipeline templates
Pipeline templates are based on pipeline description with few differences:
......@@ -163,7 +158,8 @@ the purpose of TA3-TA2 API we are currently placing the following restrictions:
* The placeholder step has to have only one input, a Dataset container value, and one output,
predictions as a Pandas dataframe. In this way it resembles a standard pipeline.
* The placeholder can be only the last step in the pipeline.
* All primitive steps should have all their hyper-parameters fixed.
* All primitive steps should have all their hyper-parameters fixed (see also `use_default_values_for_free_hyperparams`
flag to control this requirement).
These restrictions effectively mean that a pipeline template can only specify a directed acyclic graph of preprocessing
primitives that transforms one or more input Dataset container values into a *single* transformed
......@@ -172,11 +168,13 @@ There are no additional restrictions on the types of individual primitives that
pipeline template, although impact on downstream TA2 processing should be assessed before a given
primitive is used.
Relaxation: Individual systems can relax those restrictions. For example, they might allow
a placeholder step to have postprocessing primitive steps after it. In this case postprocessing
Individual systems can relax those restrictions. For example, they might allow a placeholder step to
have postprocessing primitive steps after it. In this case postprocessing
primitives can only transform predictions from a placeholder step into transformed predictions.
Or individual systems might allow primitve steps to have free hyper-parameters a TA2 system
should tune.
Or individual systems might allow primitive steps to have free hyper-parameters a TA2 system
should tune (see `use_default_values_for_free_hyperparams` flag to potentially control this behavior).
We expect that some TA2 systems will be able to work with those relaxed requirements, and
TA3s can use those if available, but it is not expected that every TA2 will.
### Fully specified pipelines
......@@ -194,10 +192,11 @@ to convert input value to closest container type value, e.g., GRPC `RAW` list va
be converted to `d3m.container.List` with generated metadata, CSV file read as Pandas
DataFrame should be converted to `d3m.container.DataFrame` with generated metadata.
Relaxation: Individual systems can support also pipelines with all primitives specified,
Individual systems can also support pipelines with all primitives specified,
but with free (available for tuning) hyper-parameters. In this case, TA2 will only tune
hyper-parameters and resulting pipelines will have the same structure as given pipeline,
but hyper-parameter configuration will differ.
but hyper-parameter configuration will differ. If such potential behavior of a system is
not desired, a `use_default_values_for_free_hyperparams` flag can be set to true.
## Values
......@@ -370,10 +369,11 @@ In Go, accessing version is slightly more involved and it is described
## Extensions of messages
GRPC and Protocol Buffers support a simple method of extending messages: just define extra fields with custom tags
in your local version of the protocol. Performers can do that to experiment with variations of the protocol (and if
changes work out, they can submit a merge request to the common API). To make sure such unofficial fields in messages
do not conflict between performers, use values from the [allocated tag ranges](./private_tag_ranges.txt) for your
organization.
in your local version of the protocol. Users for this protocol can do that to experiment with variations of the protocol (and if
changes work out, they can submit a merge request for those changes to be included into this specification).
To make sure such unofficial fields in messages do not conflict between performers, use values from the
[allocated tag ranges](./private_tag_ranges.txt) for your organization, or add your organization via a
merge request.
## Changelog
......
......@@ -39,7 +39,7 @@ setup(
author='DARPA D3M Program',
packages=find_packages(exclude=['contrib', 'docs', 'tests*']),
install_requires=[
'd3m==2020.1.9',
'd3m==2020.5.18',
'grpcio',
'grpcio-tools',
],
......
__version__ = '2020.2.11'
__version__ = '2020.6.2'
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
......@@ -2,8 +2,6 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: primitive.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
......@@ -19,8 +17,8 @@ DESCRIPTOR = _descriptor.FileDescriptor(
name='primitive.proto',
package='',
syntax='proto3',
serialized_options=_b('Z\010pipeline'),
serialized_pb=_b('\n\x0fprimitive.proto\"[\n\tPrimitive\x12\n\n\x02id\x18\x01 \x01(\t\x12\x0f\n\x07version\x18\x02 \x01(\t\x12\x13\n\x0bpython_path\x18\x03 \x01(\t\x12\x0c\n\x04name\x18\x04 \x01(\t\x12\x0e\n\x06\x64igest\x18\x05 \x01(\tB\nZ\x08pipelineb\x06proto3')
serialized_options=b'Z\010pipeline',
serialized_pb=b'\n\x0fprimitive.proto\"[\n\tPrimitive\x12\n\n\x02id\x18\x01 \x01(\t\x12\x0f\n\x07version\x18\x02 \x01(\t\x12\x13\n\x0bpython_path\x18\x03 \x01(\t\x12\x0c\n\x04name\x18\x04 \x01(\t\x12\x0e\n\x06\x64igest\x18\x05 \x01(\tB\nZ\x08pipelineb\x06proto3'
)
......@@ -36,35 +34,35 @@ _PRIMITIVE = _descriptor.Descriptor(
_descriptor.FieldDescriptor(
name='id', full_name='Primitive.id', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='version', full_name='Primitive.version', index=1,
number=2, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='python_path', full_name='Primitive.python_path', index=2,
number=3, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='name', full_name='Primitive.name', index=3,
number=4, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='digest', full_name='Primitive.digest', index=4,
number=5, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
has_default_value=False, default_value=b"".decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
......
This diff is collapsed.
......@@ -65,17 +65,17 @@ class ValueType(d3m_utils.Enum):
"""
Enumeration of possible value types.
Values are kept in sync with TA2-TA3 API's ``ValueType`` enumeration.
Values are kept in sync with standard TA2-TA3 API's value types.
"""
RAW = 1
DATASET_URI = 2
CSV_URI = 3
PICKLE_URI = 4
PICKLE_BLOB = 5
PLASMA_ID = 6
LARGE_RAW = 7
LARGE_PICKLE_BLOB = 8
RAW = 'RAW'
DATASET_URI = 'DATASET_URI'
CSV_URI = 'CSV_URI'
PICKLE_URI = 'PICKLE_URI'
PICKLE_BLOB = 'PICKLE_BLOB'
PLASMA_ID = 'PLASMA_ID'
LARGE_RAW = 'LARGE_RAW'
LARGE_PICKLE_BLOB = 'LARGE_PICKLE_BLOB'
def _can_encode_raw(value):
......@@ -235,7 +235,11 @@ def validate_uri(uri, data_directories=None):
normalized_path = os.path.normpath(parsed_uri.path)
if data_directories is not None:
if not any(os.path.commonpath([normalized_path, data_directory]) == data_directory for data_directory in data_directories):
for data_directory in data_directories:
data_directory = os.path.normpath(data_directory)
if os.path.commonpath([normalized_path, data_directory]) == data_directory:
break
else:
raise exceptions.InvalidArgumentValueError(
"URI '{uri}' (path '{normalized_path}') outside data directories: {data_directories}".format(
uri=uri,
......@@ -953,7 +957,7 @@ def decode_performance_metric(metric):
try:
metric_value = problem_module.PerformanceMetric(metric.metric)
except ValueError:
metric_value = problem_pb2.PerformanceMetric.Name(metric.metric)
metric_value = metric.metric
decoded_metric = {
'metric': metric_value,
......
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment