Parquet to TimeSeries TimeArray
Hello,
I'm reading a Parquet file which was previously saved with Python Pandas DataFrame and try to open it with Julia as TimeSeries TimeArray.
I noticed
ds = Parquet2.Dataset("/path/to/file")
dfc = ds |> TableOperations.select(:time, :open, :high, :low, :close, :volume) |> DataFrame
can help to convert to DataFrame but
using TimeSeries
ds |> TableOperations.select(:time, :open, :high, :low, :close, :volume) |> TimeArray
raises
ERROR: UndefKeywordError: keyword argument `timestamp` not assigned
Stacktrace:
[1] TimeArray(x::TableOperations.Select{Tables.CopiedColumns{NamedTuple{(:time, :open, :high, :low, :close, :volume, :close_time, :quote_asset_volume, :number_of_trades, :taker_buy_base_asset_volume, :taker_buy_quote_asset_volume, :ignore), Tuple{Vector{Union{Missing, DateTime}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, DateTime}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Int64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}}}}, true, (:time, :open, :high, :low, :close, :volume)})
@ TimeSeries.TablesIntegration C:\Users\scell\.julia\packages\TimeSeries\KZKvJ\src\tables.jl:71
[2] |>(x::TableOperations.Select{Tables.CopiedColumns{NamedTuple{(:time, :open, :high, :low, :close, :volume, :close_time, :quote_asset_volume, :number_of_trades, :taker_buy_base_asset_volume, :taker_buy_quote_asset_volume, :ignore), Tuple{Vector{Union{Missing, DateTime}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, DateTime}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Int64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}, Vector{Union{Missing, Float64}}}}}, true, (:time, :open, :high, :low, :close, :volume)}, f::Type{TimeArray})
@ Base .\operators.jl:907
[3] top-level scope
@ REPL[108]:1
What is the most appropriate way of converting to TimeSeries ?
TimeArray(ds |> TableOperations.select(:time, :open, :high, :low, :close, :volume), timestamp=:time)
but is odd as it mixes function chaining operator (|>) and function call.
So maybe something like
ds |> TableOperations.select(:time, :open, :high, :low, :close, :volume) |> t -> TimeArray(t, timestamp=:time)
looks better
Isn't there a way for select to behave as SQL alias (SELECT column1 AS c1, column2 FROM table) ?
A nicer API call could be
ds |> TableOperations.select(:time => :timestamp, :open, :high, :low, :close, :volume) |> TimeArray
Any opinion? https://github.com/JuliaData/TableOperations.jl/issues/33
Kind regards
Edited by femtotrader