Aug-17-2018, 07:28 PM
We have pyarrow 0.9 installed.
Parquet-cpp 1.4.1 is bundled with it.
We write parquet files all okay to AWS S3.
There is a Hive database with an external table overlay over the target parquet folder.
The Hive database has parquet format 1.8.1 installed.
Queries work okay.
However, we get warning messages due to the Parquet version differences.
How to remove the warning messages ?
How can we upgrade parquet-cpp to version 1.8.1 to match the hive parquet format version.
or
Is there another approach we should consider ?
After many types of searches, the solution is not found.
Any help is appreciated.
Many Thanks - DP
Parquet-cpp 1.4.1 is bundled with it.
We write parquet files all okay to AWS S3.
There is a Hive database with an external table overlay over the target parquet folder.
The Hive database has parquet format 1.8.1 installed.
Queries work okay.
However, we get warning messages due to the Parquet version differences.
How to remove the warning messages ?
How can we upgrade parquet-cpp to version 1.8.1 to match the hive parquet format version.
or
Is there another approach we should consider ?
After many types of searches, the solution is not found.
Any help is appreciated.
Many Thanks - DP