两套集群,一套为刚搭建的 Ozone,一套为已有的 Hadoop 集群,进行对接测试,让 Hadoop 使用 Ozone 对象存储文件系统作为后端存储。


Install Ozone

$ wget https://mirrors.tuna.tsinghua.edu.cn/apache/ozone/1.1.0/ozone-1.1.0.tar.gz
$ tar -zxvf ozone-1.1.0.tar.gz
# 生成配置模版 ozone-site.xml
$ ozone genconf $OZONE_HOME/etc/hadoop 
$ vim $OZONE_HOME/etc/hadoop/ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
        <tag>OM, REQUIRED</tag>
        <description>The address of the Ozone OM service. This allows clients to discoverthe address of the OM.</description>
      	  This setting is the fallback location for SCM, OM, Recon and DataNodes
      	  to store their metadata. This setting may be used only in test/PoC
      	  clusters to simplify configuration.

      	  For production clusters or any time you care about performance, it is
      	  recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
      	  dfs.container.ratis.datanode.storage.dir be configured separately.
        <tag>OZONE, SCM, REQUIRED</tag>
      	  The address of the Ozone SCM client service. This is a required setting.

      	  It is a string in the host:port format. The port number is optional
      	  and defaults to 9860.
        <tag>OZONE, REQUIRED</tag>
      	  The value of this property is a set of DNS | DNS:PORT | IPAddress | IP:PORT.
	      Written as a comma separated string. e.g. scm1,scm2:8020,
      	  This property allows datanodes to discover where SCM is, so that
      	  datanodes can send heartbeat to SCM.
        <tag>OZONE, CLIENT</tag>
          Default replication type to be used while writing key into ozone. The
          value can be specified when writing the key, default is used when
          nothing is specified. Supported values: RATIS, STAND_ALONE and CHAINED.
        <description>Boolean value to enable or disable SCM safe mode.</description>
$ ozone scm --init
# Start SCM
$ ozone --daemon start scm

SCM UI: http://localhost:9876,由参数ozone.scm.http-address控制

$ ozone om --init
# Start OM
$ ozone --daemon start om

OM UI: http://localhost:9874,由参数ozone.om.http-address控制

# Execute on every datanode
$ ozone --daemon start datanode

# Create a volume and bucket if you use o3fs scheme.
$ ozone sh volume create /bigdata
$ ozone sh bucket create /bigdata/hdfs

# Upload to key `test.txt` to /bigdata/hdfs
$ ozone sh key put /bigdata/hdfs/test.txt test.txt
# Download key `test.txt` from /bigdata/hdfs
$ ozone sh key get /volume/bucket/test.txt ./

Modify Hadoop

$ vim $HADOOP_HOME/etc/hadoop/core-site.xml

<!-- o3fs configuration -->
    <!-- hdfs namenode service -->
    <!-- <value>hdfs://localhost:9000</value> -->
    <!-- o3fs://bucket.volume -->

<!-- ofs configuration -->

# 添加依赖,Hadoop 3.x 改成 hadoop3
export HADOOP_CLASSPATH=$OZONE_HOME/share/ozone/lib/hadoop-ozone-filesystem-hadoop2-*.jar:$HADOOP_CLASSPATH
# 或者
$ cp $OZONE_HOME/share/ozone/lib/hadoop-ozone-filesystem-hadoop2-*.jar $HADOOP_HOME/share/hadoop/common/

# 将 ozone-site.xml 拷贝到 HADOOP_CONF_DIR
$ cp $OZONE_HOME/etc/hadoop/ozone-site.xml $HADOOP_HOME/etc/hadoop

# 重启 HDFS 服务
$ $HADOOP_HOM/sbin/start-dfs.sh

$ hadoop fs -ls /

Note: Ozone supports two scheme: o3fs:// and ofs://. The biggest difference between the o3fs and ofs,is that o3fs supports operations only at a single bucket, while ofs supports operations across all volumes and buckets and provides a full view of all the volume/buckets.


$ ozone --daemon start recon

Recon UI: http://localhost:9888

S3 compatible REST gateway

$ ozone --daemon start s3g