Skip to content

WIP: Add a primitive that transforms key-value pair into columns

Mingjie Sun requested to merge key-value-to-column into master

Write a primitive that transforms key-value pair in a table into columns.

The primitive is based on comparing column names between two tables and detecting whether the table contains key/value columns. This primitive will compare every table with the main resource and decide if the current table is a key/value column.

This primitive is tested against the world-development-dataset ( https://www.kaggle.com/worldbank/world-development-indicators ). There is also unit-test code in /tests/test_key_value_to_column.py.

About the implementation of the primitive:

  1. _is_transform function decide if a table is a key/value table and if we detect that this table contains key/value columns, then return the indexes of the key/value columns.

  2. _tranform function will do the actual transformation. It will make each distinct value in the key column a separate row.

  3. One thing to note is that the detection of such key/value tables is hard-cored.

Edited by Remi Rampin

Merge request reports