1.Hive简介 起源自facebook由Jeff Hammerbacher领导的团队 构建在Hadoop上的数据仓库框架 设计目的是让SQL技能良好，但Java技能较弱的分析师可以查询海量数据 2008年facebook把hive项目贡献给Apache Hive的组件与体系架构 用户接口：shell, thrift, web等 Th
1.Hive简介
起源自facebook由Jeff Hammerbacher领导的团队
构建在Hadoop上的数据仓库框架
设计目的是让SQL技能良好，但Java技能较弱的分析师可以查询海量数据
2008年facebook把hive项目贡献给Apache
Hive的组件与体系架构
用户接口：shell, thrift, web等
Thrift服务器
元数据库“Derby, Mysql等
解析器
Hadoop
Hive安装模式
内嵌模式：元数据保持在内嵌的Derby模式，只允许一个会话连接(默认)
本地独立模式：在本地安装Mysql，把元数据放到Mysql内
远程模式：元数据放置在远程的Mysql数据库
2.配置Mysql
1:copy mysql-connector-java-5.1.6-bin.jar到$HIVE_HOME/lib
[jifeng@jifeng02 hadoop]$ ls 7287OS_Code hadoop-1.2.1.tar.gz hive-0.12.0-bin tmp hadoop-1.2.1 hadoop-2.4.1-src.tar.gz hive-0.12.0-bin.tar.gz hadoop-1.2.1-bin.tar.gz hadoop-2.4.1.tar.gz mysql-connector-java-5.1.6-bin.jar [jifeng@jifeng02 hadoop]$ cp mysql-connector-java-5.1.6-bin.jar hive-0.12.0-bin/lib
2:修改$HIVE_HOME/conf/hive-site.xml
javax.jdo.option.ConnectionURL jdbc:mysql://jifengsql:3306/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.PersistenceManagerFactoryClass org.datanucleus.api.jdo.JDOPersistenceManagerFactory class implementing the jdo persistence javax.jdo.option.DetachAllOnCommit true detaches all objects from session so that they can be used after transaction is committed javax.jdo.option.NonTransactionalRead true reads outside of transactions javax.jdo.option.ConnectionUserName dss username to use against metastore database javax.jdo.option.ConnectionPassword jifeng password to use against metastore database
3:启动hive
[jifeng@jifeng02 hive-0.12.0-bin]$ hive Logging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.properties hive> show tables; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient hive> quit;报错，网上查询后说是没把 mysql的jar包mysql-connector-java-5.1.10-bin.jar，放在hive安装目录的lib下
把 mysql-connector-java-5.1.6-bin.jar 替换成mysql-connector-java-5.1.10-bin.jar还是不行。
检查 mysql发现连接不上，换个虚拟机上mysql
[dss@localhost ~]$ mysql -u root -p Enter password: --root 登陆mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 70 Server version: 5.6.16 MySQL Community Server (GPL) Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> create database hive; Query OK, 1 row affected (0.01 sec) --创建hive库 mysql> GRANT all ON hive.* TO dss@'%' IDENTIFIED BY 'abc123'; Query OK, 0 rows affected (0.03 sec)--给dss用户对hive库授权 mysql> flush privileges; Query OK, 0 rows affected (0.02 sec)--刷新系统权限表 mysql> set globalbinlog_format='MIXED'; ERROR 1193 (HY000): Unknown system variable 'globalbinlog_format' mysql> alter database hive character set latin1 ; Query OK, 1 row affected (0.00 sec)--更新字符集
再次启动
[jifeng@jifeng02 hive-0.12.0-bin]$ hive Logging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.properties hive> show tables; OK Time taken: 6.273 seconds hive>
没有错误了。
4.Hive的运行模式即任务的执行环境
1 启动 hive 命令行模式：
1 ：直接输入 #hive 的执行程序，
2 ：或者输入 #hive --service cli
l 分为本地与集群两种
我们可以通过 mapred.job.tracker 来指明
设置方式：
hive >
SET mapred.job.tracker=local
2.hive 验证启动的方法
l 1 、 hive web 界面的 ( 端口号 9999) 启动方式
#hive --service hwi
[jifeng@jifeng02 hive-0.12.0-bin]$ hive --service cli Logging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.properties hive> quit; [jifeng@jifeng02 hive-0.12.0-bin]$ hive --service hwi 15/08/17 15:17:10 INFO hwi.HWIServer: HWI is starting up 15/08/17 15:17:10 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 15/08/17 15:17:10 INFO mortbay.log: jetty-6.1.26 15/08/17 15:17:10 INFO mortbay.log: Extract /home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-hwi-0.12.0.war to /tmp/Jetty_0_0_0_0_9999_hive.hwi.0.12.0.war__hwi__ow27i/webapp 15/08/17 15:17:11 INFO mortbay.log: Started SocketConnector@0.0.0.0:9999
用于通过浏览器来访问 hive
http://jifeng02:9999/hwi/
l 2 、 hive 远程服务 ( 端口号 10000) 启动方式
#hive --service hiveserver