WIP: Add a primitive that transforms key-value pair into columns
Write a primitive that transforms key-value pair in a table into columns.
The primitive is based on comparing column names between two tables and detecting whether the table contains key/value columns. This primitive will compare every table with the main resource and decide if the current table is a key/value column.
This primitive is tested against the world-development-dataset ( https://www.kaggle.com/worldbank/world-development-indicators ). There is also unit-test code in /tests/test_key_value_to_column.py.
About the implementation of the primitive:
-
_is_transform
function decide if a table is a key/value table and if we detect that this table contains key/value columns, then return the indexes of the key/value columns. -
_tranform
function will do the actual transformation. It will make each distinct value in the key column a separate row. -
One thing to note is that the detection of such key/value tables is hard-cored.