topic Re: DB Connect 3 Hive Connection in All Apps and Add-ons

DB Connect 3 Hive Connection

jamesmarlowww — Tue, 29 Sep 2020 15:47:29 GMT

Hey,

We are trying to connect "DB connect 3" to our Hive DB. Unfortunately I'm running into some issues.

I have followed the instructions from this link, but I am still facing issues: https://www.splunk.com/blog/2015/02/25/splunk-db-connect-cloudera-hive-jdbc-connector.html

Our Hive instance is on version 1.2.1. I am attempting to use both the Hive 1.2.1 and 2.5.6 JDBC drivers, I get the same error from both sets of drivers.

The JDBC URL I'm using
jdbc:hive2://{IP}:{Port}/default

The errror - I have looked through the Jar file and this Class is in there
io.dropwizard.jersey.errors.LoggingExceptionMapper - Error handling a request: d5f34034d981f17d
java.lang.NoClassDefFoundError: Could not initialize class com.cloudera.hive.hive.core.Hive2JDBCDriver

My config entry in db_connection_types.conf
- attempt using different jar hive-jdbc-1.2.1.jar\org\apache\hive\jdbc\HiveDriver
[hive3]
displayName = Hive Server 3
jdbcDriverClass = org.apache.hive.jdbc.HiveDriver
defaultPort = 10001
connectionUrlFormat = jdbc:hive2://{0}:{1}/{2}
defaultSchema = default
defaultCatalogName = default

*Attempt using HiveJDBC4.jar *
[hive4]
displayName = Hive Server 4
dbcDriverClass = com.cloudera.hive.jdbc4.HS1Driver
defaultPort = 10001
connectionUrlFormat = jdbc:hive2://{0}:{1}/{2}
defaultSchema = default
defaultCatalogName = default

Thanks

James

Re: DB Connect 3 Hive Connection

rdagan_splunk — Tue, 29 Sep 2020 15:45:29 GMT

Hi James, Can you share what your driver directory looks like?
What files you have under the directory /opt/splunk/etc/apps/splunk_app_db_connect/driver ?

Re: DB Connect 3 Hive Connection

jamesmarlowww — Tue, 29 Sep 2020 15:48:11 GMT

Thanks for the reply. Yup sure, these are the files

-rw-r--r--. 1 splunk splunk 58160 Sep 15 12:03 commons-codec-1.4.jar
-rw-r--r--. 1 splunk splunk 575389 Sep 15 12:03 commons-collections-3.2.1.jar
-rw-r--r--. 1 splunk splunk 298829 Sep 15 12:03 commons-configuration-1.6.jar
-rw-r--r--. 1 splunk splunk 279193 Sep 15 12:03 commons-lang-2.5.jar
-rw-r--r--. 1 splunk splunk 61829 Sep 15 12:03 commons-logging-1.2.jar
-rw-r--r--. 1 splunk splunk 3866524 Mar 15 2017 db2jcc4.jar
-rw-r--r--. 1 splunk splunk 1648200 Sep 15 12:03 guava-11.0.2.jar
-rw-r--r--. 1 splunk splunk 3431544 Sep 15 12:03 hadoop-common-2.7.1.jar
-rw-r--r--. 1 splunk splunk 292289 Sep 15 12:03 hive-common-1.2.1.jar
-rw-r--r--. 1 splunk splunk 100580 Sep 15 12:03 hive-jdbc-1.2.1.jar
-rw-r--r--. 1 splunk splunk 1511661 Sep 15 14:22 HiveJDBC4.jar
-rw-r--r--. 1 splunk splunk 1511661 Sep 14 14:22 HiveJDBC4.jar.1.0
-rw-r--r--. 1 splunk splunk 5505100 Sep 15 12:03 hive-metastore-1.2.1.jar
-rw-r--r--. 1 splunk splunk 7670596 Sep 15 14:22 hive_metastore.jar
-rw-r--r--. 1 splunk splunk 1878543 Sep 15 12:03 hive-service-1.2.1.jar
-rw-r--r--. 1 splunk splunk 596600 Sep 15 14:22 hive_service.jar
-rw-r--r--. 1 splunk splunk 727567 Sep 15 12:03 httpclient-4.5.jar
-rw-r--r--. 1 splunk splunk 322234 Sep 15 12:03 httpcore-4.4.1.jar
-rw-r--r--. 1 splunk splunk 275186 Sep 15 14:22 libfb303-0.9.0.jar
-rw-r--r--. 1 splunk splunk 347531 Sep 15 14:22 libthrift-0.9.0.jar
-rw-r--r--. 1 splunk splunk 367444 Sep 15 14:22 log4j-1.2.14.jar
-rw-r--r--. 1 splunk splunk 489884 Sep 15 12:03 log4j-1.2.17.jar
drwxr-xr-x. 4 splunk splunk 4096 May 4 2016 mysql-connector-java-5.1.39
-rw-r--r--. 1 splunk splunk 989497 Mar 15 2017 mysql-connector-java-5.1.39-bin.jar
-rw-r--r--. 1 splunk splunk 359276 Mar 15 2017 nzjdbc3.jar
-rw-r--r--. 1 splunk splunk 3698857 Mar 15 2017 ojdbc7.jar
-rw-r--r--. 1 splunk splunk 707261 Mar 15 2017 postgresql-42.0.0.jar
-rw-r--r--. 1 splunk splunk 294796 Sep 15 14:22 ql.jar
-rw-r--r--. 1 splunk splunk 41 Jul 5 17:38 README
-rw-r--r--. 1 splunk splunk 23671 Sep 15 14:22 slf4j-api-1.5.11.jar
-rw-r--r--. 1 splunk splunk 32119 Sep 15 12:03 slf4j-api-1.7.10.jar
-rw-r--r--. 1 splunk splunk 9693 Sep 15 14:22 slf4j-log4j12-1.5.11.jar
-rw-r--r--. 1 splunk splunk 8869 Sep 15 12:03 slf4j-log4j12-1.7.5.jar
-rw-r--r--. 1 splunk splunk 653996 Mar 16 2017 sqljdbc42.jar
-rw-r--r--. 1 splunk splunk 1307923 Sep 15 14:22 TCLIServiceClient.jar

Re: DB Connect 3 Hive Connection

gwobben — Tue, 03 Oct 2017 13:15:42 GMT

any luck fixing this?

Re: DB Connect 3 Hive Connection

jamesmarlowww — Wed, 04 Oct 2017 20:33:10 GMT

Nope no luck 😞

Re: DB Connect 3 Hive Connection

rdagan_splunk — Tue, 29 Sep 2020 16:07:27 GMT

1)
Based on this: http://www.cloudera.com/documentation/other/connectors/hive-jdbc/2-5-6/Cloudera-JDBC-Driver-for-Apache-Hive-Install-Guide-2-5-6.pdf
It looks like Hive version 1.2.1 is not supported.

2)
For DBX 3, It looks like you are missing a directory.
Normally I see: only one jar under /opt/splunk/etc/apps/splunk_app_db_connect/driver
For example hive-jdbc-1.2.1.jar
And under this sub directory you will put all the rest of the supporting jar files
/opt/splunk/etc/apps/splunk_app_db_connect/driver/hive-jdbc-1.2.1-lib

3)
See the setup for Spark and Teradata
http://docs.splunk.com/Documentation/DBX/3.1.1/DeployDBX/Installdatabasedrivers#Spark_SQL