hive 的安装与使用

在本篇文章中将使用mysql储存hive的元数据,hive版本为2.3.3
官方下载页链接:http://hive.apache.org/downlo...
官方DDL文档页链接:https://cwiki.apache.org/conf...
先安装mysql服务以备后用,这里不做介绍,安装后启动即可

# 下载hive
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.3/apache-hive-2.3.3-bin.tar.gz
# 解压缩
tar -zxvf apache-hive-2.3.3-bin.tar.gz -C /usr/local
# 创建配置文件
touch hive-site.xml

修改配置文件hive-env.sh

# 修改hadoop安装路径
HADOOP_HOME=/usr/local/hadoop

修改配置文件hive-site.xml

<configuration>
  <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
  <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
  <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
  <!-- Hive Execution Parameters -->
  <!-- mysql作元数据储存数据库 -->
  <property>
   <name>javax.jdo.option.ConnectionURL</name>
   <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
  </property>
  <property>
   <name>javax.jdo.option.ConnectionDriverName</name>
   <value>com.mysql.jdbc.Driver</value>
  </property>
  <property>
   <name>javax.jdo.option.ConnectionUserName</name>
   <value>root</value>
  </property>
  <property>
   <name>javax.jdo.option.ConnectionPassword</name>
   <value>123456</value>
  </property>
</configuration>

将mysql连接器驱动放入hive安装目录下的lib目录中,这里要注意别用最新的驱动
博主这里用的是5.1
驱动下载页面链接:https://dev.mysql.com/downloa...
初始化元数据到mysql数据库

/us/local/hive/bin/schematool -initSchema -dbType mysql

查看数据库出现了hive库和一堆元数据表
接着就可以启动hive了

/usr/local/hive/bin/hive

至此一切ok

而外介绍

# 创建表事例
CREATE TABLE IF NOT EXISTS test (
id int
,uid int
,title string
,name string
,status int
,time timestamp)
COMMENT '简介'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY "\001"
LINES TERMINATED BY "\n"
STORED AS TEXTFILE;

CREATE TABLE IF NOT EXISTS test_out (
name string
, count int
,time date)
COMMENT '简介'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\001'
LINES TERMINATED BY '\n'
STORED AS TEXTFILE;

# 统计后将结果数据加入另一个表
INSERT INTO TABLE 
test_out(name,count,time) 
SELECT name,count(1),to_date(time) 
FROM test 
GROUP BY name,to_date(time);

INSERT OVERWRITE 
TABLE test_out
SELECT name,count(1),to_date(time) 
FROM test 
GROUP BY name,to_date(time);

评论

暂无评论