vmuser@vmuser-VirtualBox:~$ sudo useradd -m hadoop -s /bin/bash
[sudo] vmuser 的密码: vmuser@vmuser-VirtualBox:~$ sudo passwd hadoop输入新的 UNIX 密码: 重新输入新的 UNIX 密码: passwd:已成功更新密码vmuser@vmuser-VirtualBox:~$ sudo adduser hadoop sudo 正在添加用户"hadoop"到"sudo"组...正在将用户“hadoop”加入到“sudo”组中完成。vmuser@vmuser-VirtualBox:~$ sudo apt-get install openssh-server正在读取软件包列表... 完成正在分析软件包的依赖关系树 正在读取状态信息... 完成 将会同时安装下列软件: ncurses-term openssh-client openssh-sftp-server ssh-import-id建议安装: keychain libpam-ssh monkeysphere ssh-askpass molly-guard rssh下列【新】软件包将被安装: ncurses-term openssh-server openssh-sftp-server ssh-import-id下列软件包将被升级: openssh-client升级了 1 个软件包,新安装了 4 个软件包,要卸载 0 个软件包,有 386 个软件包未被升级。需要下载 637 kB/1,248 kB 的归档。解压缩后会消耗 5,316 kB 的额外空间。您希望继续执行吗? [Y/n] y获取:1 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 ncurses-term all 6.1-1ubuntu1.18.04 [248 kB]获取:2 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 openssh-sftp-server amd64 1:7.6p1-4ubuntu0.1 [45.5 kB]获取:3 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 openssh-server amd64 1:7.6p1-4ubuntu0.1 [332 kB]获取:4 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 ssh-import-id all 5.7-0ubuntu1.1 [10.9 kB]已下载 637 kB,耗时 0秒 (3,215 kB/s) 正在预设定软件包 ...(正在读取数据库 ... 系统当前共安装有 140635 个文件和目录。)正准备解包 .../openssh-client_1%3a7.6p1-4ubuntu0.1_amd64.deb ...正在将 openssh-client (1:7.6p1-4ubuntu0.1) 解包到 (1:7.6p1-4) 上 ...正在选中未选择的软件包 ncurses-term。正准备解包 .../ncurses-term_6.1-1ubuntu1.18.04_all.deb ...正在解包 ncurses-term (6.1-1ubuntu1.18.04) ...正在选中未选择的软件包 openssh-sftp-server。正准备解包 .../openssh-sftp-server_1%3a7.6p1-4ubuntu0.1_amd64.deb ...正在解包 openssh-sftp-server (1:7.6p1-4ubuntu0.1) ...正在选中未选择的软件包 openssh-server。正准备解包 .../openssh-server_1%3a7.6p1-4ubuntu0.1_amd64.deb ...正在解包 openssh-server (1:7.6p1-4ubuntu0.1) ...正在选中未选择的软件包 ssh-import-id。正准备解包 .../ssh-import-id_5.7-0ubuntu1.1_all.deb ...正在解包 ssh-import-id (5.7-0ubuntu1.1) ...正在设置 ncurses-term (6.1-1ubuntu1.18.04) ...正在处理用于 ufw (0.35-5) 的触发器 ...正在处理用于 ureadahead (0.100.0-20) 的触发器 ...正在处理用于 systemd (237-3ubuntu10.3) 的触发器 ...正在处理用于 man-db (2.8.3-2) 的触发器 ...正在设置 openssh-client (1:7.6p1-4ubuntu0.1) ...正在设置 ssh-import-id (5.7-0ubuntu1.1) ...正在设置 openssh-sftp-server (1:7.6p1-4ubuntu0.1) ...正在设置 openssh-server (1:7.6p1-4ubuntu0.1) ...Creating config file /etc/ssh/sshd_config with new versionCreating SSH2 RSA key; this may take some time ...2048 SHA256:+UzKAYam4shz8eFVxXfjzNthoobWMAQk/ncyjqOwnAY root@vmuser-VirtualBox (RSA)Creating SSH2 ECDSA key; this may take some time ...256 SHA256:Y2lX2D5tkCgVFTgJ6inqzO0v5o07tgEhD2sgea8w24M root@vmuser-VirtualBox (ECDSA)Creating SSH2 ED25519 key; this may take some time ...256 SHA256:ZcP+F60Vdlb/ddaHog4J/KhZFAgufj4IRyxnnqrUtgM root@vmuser-VirtualBox (ED25519)Created symlink /etc/systemd/system/sshd.service → /lib/systemd/system/ssh.service.Created symlink /etc/systemd/system/multi-user.target.wants/ssh.service → /lib/systemd/system/ssh.service.正在处理用于 ureadahead (0.100.0-20) 的触发器 ...正在处理用于 systemd (237-3ubuntu10.3) 的触发器 ...hadoop@vmuser-VirtualBox:/home/vmuser$ sudo su hadoop[sudo] hadoop 的密码: hadoop@vmuser-VirtualBox:/home/vmuser$ cd ~hadoop@vmuser-VirtualBox:~$ pwd/home/hadoophadoop@vmuser-VirtualBox:~$ lsexamples.desktophadoop@vmuser-VirtualBox:~$ lsexamples.desktophadoop@vmuser-VirtualBox:~$ mkdir .sshhadoop@vmuser-VirtualBox:~$ cd .ssh/hadoop@vmuser-VirtualBox:~/.ssh$ ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/hadoop/.ssh/id_rsa.Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.The key fingerprint is:SHA256:VjU4i/ypqi/K4xRQXmYN1wX2JyE0o23ehNR3W+4PSf8 hadoop@vmuser-VirtualBoxThe key's randomart image is:+---[RSA 2048]----+| . =o.oO+o.o || o + ..=.*+o... .||. . ..+.=+.. + || . oo+.o o .|| . S... . + || . . o o o|| . . .o|| o. . . E|| .+o.+o. |+----[SHA256]-----+hadoop@vmuser-VirtualBox:~/.ssh$ cat id_rsa.pub >> authorized_keyshadoop@vmuser-VirtualBox:~/.ssh$ sudo apt install openjdk-8-jdk openjdk-8-jre正在读取软件包列表... 完成正在分析软件包的依赖关系树 正在读取状态信息... 完成 update-alternatives: 使用 /usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole 来在自动模式中提供 /usr/bin/jconsole (jconsole)正在处理用于 libc-bin (2.27-3ubuntu1) 的触发器 ...正在处理用于 ca-certificates (20180409) 的触发器 ...Updating certificates in /etc/ssl/certs...0 added, 0 removed; done.Running hooks in /etc/ca-certificates/update.d...done.done.hadoop@vmuser-VirtualBox:~/.ssh$ dpkg -L openjdk-8-jdk /./usr/usr/lib/usr/lib/jvm/usr/lib/jvm/java-8-openjdk-amd64/usr/lib/jvm/java-8-openjdk-amd64/bin/usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer/usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole/usr/lib/jvm/java-8-openjdk-amd64/include/usr/lib/jvm/java-8-openjdk-amd64/include/jawt.h/usr/lib/jvm/java-8-openjdk-amd64/include/linux/usr/lib/jvm/java-8-openjdk-amd64/include/linux/jawt_md.h/usr/lib/jvm/java-8-openjdk-amd64/lib/usr/lib/jvm/java-8-openjdk-amd64/lib/jconsole.jar/usr/lib/jvm/java-8-openjdk-amd64/man/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/man1/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/man1/appletviewer.1.gz/usr/lib/jvm/java-8-openjdk-amd64/man/ja_JP.UTF-8/man1/jconsole.1.gz/usr/lib/jvm/java-8-openjdk-amd64/man/man1/usr/lib/jvm/java-8-openjdk-amd64/man/man1/appletviewer.1.gz/usr/lib/jvm/java-8-openjdk-amd64/man/man1/jconsole.1.gz/usr/share/usr/share/doc/usr/share/doc/openjdk-8-jre-headless/usr/share/doc/openjdk-8-jre-headless/test-amd64/usr/share/doc/openjdk-8-jre-headless/test-amd64/check-hotspot-hotspot.log.gz/usr/share/doc/openjdk-8-jre-headless/test-amd64/check-jdk-hotspot.log.gz/usr/share/doc/openjdk-8-jre-headless/test-amd64/check-langtools-hotspot.log.gz/usr/share/doc/openjdk-8-jre-headless/test-amd64/failed_tests-hotspot.tar.gz/usr/share/doc/openjdk-8-jre-headless/test-amd64/jtreg-summary-hotspot.log.gz/usr/share/doc/openjdk-8-jre-headless/test-amd64/jtreport-hotspot.tar.gz/usr/share/doc/openjdk-8-jdkhadoop@vmuser-VirtualBox:~/.ssh$ which javac /usr/bin/javachadoop@vmuser-VirtualBox:~/.ssh$ ls -al /usr/bin/javac lrwxrwxrwx 1 root root 23 1月 20 13:29 /usr/bin/javac -> /etc/alternatives/javachadoop@vmuser-VirtualBox:~/.ssh$ ls -al /etc/alternatives/javaclrwxrwxrwx 1 root root 43 1月 20 13:29 /etc/alternatives/javac -> /usr/lib/jvm/java-8-openjdk-amd64/bin/javachadoop@vmuser-VirtualBox:~/.ssh$add in ~/.bashrc
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64hadoop@vmuser-VirtualBox:~/.ssh$ source ~/.bashrc
hadoop@vmuser-VirtualBox:~/.ssh$ echo $JAVA_HOME/usr/lib/jvm/java-8-openjdk-amd64hadoop@vmuser-VirtualBox:~/.ssh$ java -versionopenjdk version "1.8.0_191"OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)hadoop@vmuser-VirtualBox:~/.ssh$ $JAVA_HOME/bin/java -versionopenjdk version "1.8.0_191"OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)hadoop@vmuser-VirtualBox:~/.ssh$ wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz
--2019-01-20 13:38:58-- http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz正在解析主机 mirrors.hust.edu.cn (mirrors.hust.edu.cn)... 202.114.18.160正在连接 mirrors.hust.edu.cn (mirrors.hust.edu.cn)|202.114.18.160|:80... 已连接。已发出 HTTP 请求,正在等待回应... 200 OK长度: 216745683 (207M) [application/octet-stream]正在保存至: “hadoop-2.7.6.tar.gz”hadoop-2.7.6.tar.gz 100%[===================>] 206.70M 8.81MB/s 用时 30s 2019-01-20 13:39:28 (6.96 MB/s) - 已保存 “hadoop-2.7.6.tar.gz” [216745683/216745683])hadoop@vmuser-VirtualBox:/usr/local$ sudo tar zvxf hadoop-2.7.6.tar.gz -C /usr/local
hadoop@vmuser-VirtualBox:~/.ssh$ cd /usr/local/hadoop@vmuser-VirtualBox:/usr/local$ lsbin etc games hadoop-2.7.6 include lib man sbin share srchadoop@vmuser-VirtualBox:/usr/local$ sudo mv ./hadoop-2.7.6 ./hadoophadoop@vmuser-VirtualBox:/usr/local$ sudo chown -R hadoop ./hadoophadoop@vmuser-VirtualBox:/usr/local$ vim hadoop/etc/hadoop/core-site.xml hadoop@vmuser-VirtualBox:/usr/local$ vim hadoop/etc/hadoop/hdfs-site.xml hadoop@vmuser-VirtualBox:/usr/local$ cd hadoop/hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs namenode -format19/01/20 13:46:18 INFO namenode.FSImageFormatProtobuf: Saving image file /usr/local/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression19/01/20 13:46:18 INFO namenode.FSImageFormatProtobuf: Image file /usr/local/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 323 bytes saved in 0 seconds.19/01/20 13:46:18 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 019/01/20 13:46:18 INFO util.ExitUtil: Exiting with status 019/01/20 13:46:18 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************SHUTDOWN_MSG: Shutting down NameNode at vmuser-VirtualBox/127.0.1.1************************************************************/adoop@vmuser-VirtualBox:/usr/local/hadoop$ ./sbin/start-dfs.sh Starting namenodes on [localhost]The authenticity of host 'localhost (127.0.0.1)' can't be established.ECDSA key fingerprint is SHA256:Y2lX2D5tkCgVFTgJ6inqzO0v5o07tgEhD2sgea8w24M.Are you sure you want to continue connecting (yes/no)? yeslocalhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.localhost: Error: JAVA_HOME is not set and could not be found.localhost: Error: JAVA_HOME is not set and could not be found.hadoop@vmuser-VirtualBox:/usr/local/hadoop$ vim etc/hadoop/hadoop-env.sh +25etc/hadoop/hadoop-env.sh:25:export JAVA_HOME=${JAVA_HOME}
etc/hadoop/hadoop-env.sh:25:export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./sbin/start-dfs.sh Starting namenodes on [localhost]localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hadoop-namenode-vmuser-VirtualBox.outlocalhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hadoop-datanode-vmuser-VirtualBox.outStarting secondary namenodes [0.0.0.0]The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.ECDSA key fingerprint is SHA256:Y2lX2D5tkCgVFTgJ6inqzO0v5o07tgEhD2sgea8w24M.Are you sure you want to continue connecting (yes/no)? yes0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hadoop-secondarynamenode-vmuser-VirtualBox.outhadoop@vmuser-VirtualBox:/usr/local/hadoop$ jps8835 NameNode9365 Jps9241 SecondaryNameNode9019 DataNodeaccess localhost:50070
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -mkdir -p /user/hadoop
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -put ./etc/hadoop/capacity-scheduler.xml inputput: Cannot create file/user/hadoop/input/capacity-scheduler.xml._COPYING_. Name node is in safe mode.hadoop@vmuser-VirtualBox:/usr/local/hadoop$ jps15058 DataNode15574 Jps14126 NameNode14542 SecondaryNameNodehadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -put ./etc/hadoop/capacity-scheduler.xml inputput: Cannot create file/user/hadoop/input/capacity-scheduler.xml._COPYING_. Name node is in safe mode.hadoop@vmuser-VirtualBox:/usr/local/hadoop$ bin/hadoop dfsadmin -safemode leaveDEPRECATED: Use of this script to execute hdfs command is deprecated.Instead use the hdfs command for it.Safe mode is OFF
hadoop@vmuser-VirtualBox:/usr/local/hadoop$ ./bin/hdfs dfs -put ./etc/hadoop/capacity-scheduler.xml input