Oct-18-2020, 03:57 AM
I am developing Hadoop File System client with python module hdfs3. My OS is CentOS 8 and IDE is eclipse. First I try to install hdfs3 with conda install command.
# conda config --add channels conda-forge
# conda config --set channel_priority strict
# conda install hdfs3
But the installation is failed. The Conda installation command can not download the package. So I use pip installation.
# pip install hdfs3
The pip installation is successful, but the python codes throw errors
# conda config --add channels conda-forge
# conda config --set channel_priority strict
# conda install hdfs3
But the installation is failed. The Conda installation command can not download the package. So I use pip installation.
# pip install hdfs3
The pip installation is successful, but the python codes throw errors
from hdfs3 import HDFileSystem hdfs = HDFileSystem(host='localhost', port=8020)The errors are shown below,
Error:Traceback (most recent call last):
File "/home/jhwang/eclipse-workspace/BigDataPythonTest/com/aaa/etl/hdfs3-test.py", line 7, in <module>
hdfs = HDFileSystem(host='localhost', port=8020)
File "/usr/python/anaconda3/lib/python3.8/site-packages/hdfs3/core.py", line 88, in __init__
self.connect()
File "/usr/python/anaconda3/lib/python3.8/site-packages/hdfs3/core.py", line 106, in connect
get_lib()
File "/usr/python/anaconda3/lib/python3.8/site-packages/hdfs3/core.py", line 670, in get_lib
from .lib import _lib as l
File "/usr/python/anaconda3/lib/python3.8/site-packages/hdfs3/lib.py", line 24, in <module>
raise ImportError("Can not find the shared library: libhdfs3.so\n"
ImportError: Can not find the shared library: libhdfs3.so
How can I download hdfs3 and libhdfs3 modules with conda install command? Any reply will be deeply thankful.