Archive

Mapping Splunk data models to Hive

New Member

Anyone know how to do this? I want to read Splunk data directly through hive, without archiving data to hadoop. Thanks.

Tags (1)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hello. I have successfully queried hive with Splunk.

https://docs.splunk.com/Documentation/Splunk/7.2.3/HadoopAnalytics/ConfigureHivepreprocessor

In a nutshell

  • you will need a license for Hadoop Analytics
  • You either use the metastore capability or you tell Splunk what datatype each Hive field
  • You tell Splunk the database and table name for Hive
  • You tell Splunk the path to the Hive data and what the db paths will look like
  • Splunk will run MUCH faster if your data has partitions

- setting up the provider can be a little bewildering if you have never done it

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Hello. I have successfully queried hive with Splunk.

https://docs.splunk.com/Documentation/Splunk/7.2.3/HadoopAnalytics/ConfigureHivepreprocessor

In a nutshell

  • you will need a license for Hadoop Analytics
  • You either use the metastore capability or you tell Splunk what datatype each Hive field
  • You tell Splunk the database and table name for Hive
  • You tell Splunk the path to the Hive data and what the db paths will look like
  • Splunk will run MUCH faster if your data has partitions

- setting up the provider can be a little bewildering if you have never done it

View solution in original post

0 Karma

New Member

Thanks a lot. will try this. Just want to make sure, the splunk data here is not archived to Hadoop. We can directly map from Hive to Splunk data model.

0 Karma

SplunkTrust
SplunkTrust

Yes you associate a virtual index with a Hive table.

0 Karma