1. Check Hadoop version on Ubuntu 18.04 or Find Hadoop Installed Version
Syntax:
hadoop version
OR
hdfs version
Output
Hadoop 3.1.1 Source code repository https://github.com/apache/hadoop -r 2b9a8c1d3a2caf1e733d57f346af3ff0d5ba529c Compiled by leftnoteasy on 2018-08-02T04:26Z Compiled with protoc 2.5.0 From source with checksum f76ac55e5b5ff0382a9f7df36a3ca5a0 This command was run using /usr/local/hadoop/share/hadoop/common/hadoop-common-3.1.1.jar
It’s clearly showing hadoop version 3.1.1
2. List Directories in HDFS on Hadoop 3.11 – Ubuntu 18.04
Syntax:
hadoop fs -ls /
Output
Found 1 items
drwxr-xr-x - hduser supergroup 0 2018-09-16 01:51 /hadoop
hduser@sachin-Inspiron-15-3567:/usr/local/hadoop/etc/hadoop$
hadoop directory is listing
This Hadoop fs command behaves like -ls, but recursively displays entries in all sub directories of a path
hdfs dfs -ls -R /
Output
drwxr-xr-x - hduser supergroup 0 2018-09-16 01:51 /hadoop drwxr-xr-x - hduser supergroup 0 2018-09-17 23:53 /sample drwxr-xr-x - hduser supergroup 0 2018-09-17 23:53 /sample/a
You can list directory by using browser GUI
click url http://localhost:9870/explorer.html#/
Click on Utilities -> Browse the file system
3. Create directory on Hadoop file system on Hadoop 3.1.1 Ubuntu 18.04
Syntax :
hdfs dfs -mkdir /<directory location with name>
Ex: Create new directory sample on hdfs
hdfs dfs -mkdir /sample hadoop fs -ls / Found 2 items drwxr-xr-x - hduser supergroup 0 2018-09-16 01:51 /hadoop drwxr-xr-x - hduser supergroup 0 2018-09-17 23:46 /sample
Above command creates sample directory at root “/” location.
4 .put
put command copies the file or directory from the local file system to the destination of hadoop file system dfs.
Syntax:
put <localSrc> <dest>
Ex: Insert text file a.txt to hadoop dfs directory /sample/
hdfs dfs -put a.txt /sample/
list inserted file on hadoop dfs
hadoop fs -ls /sample/ Found 2 items drwxr-xr-x - hduser supergroup 0 2018-09-17 23:53 /sample/a -rw-r--r-- 1 hduser supergroup 11 2018-09-18 00:55 /sample/a.txt
Read file on hdfs using cat
hdfs dfs -cat /sample/a.txt hi sachin