Mounting HDFS cluster as a block device with hadoop-fuse

Using Hadoop may quickly become very annoying if you have to navigate through the HDFS filesystem with a standard hadoop command. As a Linux user I got used to TAB-autocompletion feature, which lets me quickly and easily use my filesystem so I was really disappointed with this difficulty. Luckily – there’s a solution which eased my pain!

The solution is called FUSE. As a Hadoop user you probably have a suitable repository in your apt, so you can install hadoop-fuse this simple:

sudo apt-get install hadoop-fuse

Then you have to create a directory to which you want to mount your HDFS to, and… just mount it. Like this:

sudo mkdir /hdfs
sudo hadoop-fuse-dfs hdfs://<host>:<port> /hdfs

Now your HDFS is accessible from your local filesystem in /hdfs directory. Much easier!

Comments are closed.