Denormalization Primitive Errors on Tidy Dataset
The denorm primitive errors when run against the denorm primitive when building the relations graph.
Traceback (most recent call last):
File "../pipelinerunner/run_pipeline.py", line 47, in <module>
output = pe.execute_pipeline_file(args.pipeline_file, args.dataset_file, args.resource)
File "/pipelinerunner/pipeline_executor.py", line 317, in execute_pipeline_file
return execute_pipeline(pipeline, dataset_filenames, static_resource_path)
File "/pipelinerunner/pipeline_executor.py", line 288, in execute_pipeline
result = getattr(primitive, output.id)(**input_data)
File "/src/common-primitives/common_primitives/denormalize.py", line 109, in produce
graph = utils.build_relation_graph(inputs)
File "/src/common-primitives/common_primitives/utils.py", line 1107, in build_relation_graph
foreign_index = get_column_index_from_column_name(dataset.metadata, column_metadata['foreign_key']['column_name'], at=foreign_resource_id)
File "/src/common-primitives/common_primitives/utils.py", line 1050, in get_column_index_from_column_name
for column_index in range(inputs_metadata.query(list(at) + [metadata_base.ALL_ELEMENTS,])['dimension']['length']):
File "/usr/local/lib/python3.6/dist-packages/frozendict/__init__.py", line 29, in __getitem__
return self._dict[key]
KeyError: 'dimension'
This could be an issue with the underlying dataset rather than the primitive itself. The above error happened when processing LL1_726_TIDY_GPS_carpool_bus_service_rating_prediction.
The full pipeline submitted was
context: TESTING
name: "denorm_test"
description: "test denorm pipeline"
inputs {
name: "input 0"
}
outputs {
name: "outputs 1"
data: "steps.1.produce"
}
steps {
primitive {
primitive {
id: "f31f8c1f-d1c5-43e5-a4b2-2ae4a761ef2e"
version: "0.2.0"
python_path: "d3m.primitives.data_transformation.denormalize.Common"
name: "Denormalize datasets"
digest: "c39e3436373aed1944edbbc9b1cf24af5c71919d73bf0bb545cba0b685812df1"
}
arguments {
key: "inputs"
value {
container {
data: "inputs.0"
}
}
}
outputs {
id: "produce"
}
}
}
steps {
primitive {
primitive {
id: "4b42ce1e-9b98-4a25-b68e-fad13311eb65"
version: "0.3.0"
python_path: "d3m.primitives.data_transformation.dataset_to_dataframe.Common"
name: "Dataset to DataFrame converter"
digest: "85b946aa6123354fe51a288c3be56aaca82e76d4071c1edc13be6f9e0e100144"
}
arguments {
key: "inputs"
value {
container {
data: "steps.0.produce"
}
}
}
outputs {
id: "produce"
}
}
}