HDFS Python client based on WebHDFS REST API
Project description
# WebHDFS Python Client Implementation
WebHDFS is REST-API to HDFS. To facilitate access to WebHDFS from Python, webhdfs-py was developed. The library can easily be installed via easy_install or pip:
easy_install webhdfs
Webhdfs-py has no further dependencies and solely relies on the Python standard library. Similar to the Python os package, webhdfs-py provides basic capabilities like the creation/listing and deletion of directories on files.
## Hadoop configuration
Ensure that WebHDFS is enabled in the `hdfs-site.xml`:
Relevant properties:
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
see <http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html>
## Limitations
* Kerberos security not supported
WebHDFS is REST-API to HDFS. To facilitate access to WebHDFS from Python, webhdfs-py was developed. The library can easily be installed via easy_install or pip:
easy_install webhdfs
Webhdfs-py has no further dependencies and solely relies on the Python standard library. Similar to the Python os package, webhdfs-py provides basic capabilities like the creation/listing and deletion of directories on files.
## Hadoop configuration
Ensure that WebHDFS is enabled in the `hdfs-site.xml`:
Relevant properties:
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
see <http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html>
## Limitations
* Kerberos security not supported
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
WebHDFS-0.2.0.tar.gz
(3.7 kB
view hashes)
