Knowledge Management

Mapping Splunk data models to Hive

sabburisplunk
New Member

Anyone know how to do this? I want to read Splunk data directly through hive, without archiving data to hadoop. Thanks.

Tags (1)
0 Karma
1 Solution

burwell
SplunkTrust
SplunkTrust

Hello. I have successfully queried hive with Splunk.

https://docs.splunk.com/Documentation/Splunk/7.2.3/HadoopAnalytics/ConfigureHivepreprocessor

In a nutshell

  • you will need a license for Hadoop Analytics
  • You either use the metastore capability or you tell Splunk what datatype each Hive field
  • You tell Splunk the database and table name for Hive
  • You tell Splunk the path to the Hive data and what the db paths will look like
  • Splunk will run MUCH faster if your data has partitions

- setting up the provider can be a little bewildering if you have never done it

View solution in original post

0 Karma

burwell
SplunkTrust
SplunkTrust

Hello. I have successfully queried hive with Splunk.

https://docs.splunk.com/Documentation/Splunk/7.2.3/HadoopAnalytics/ConfigureHivepreprocessor

In a nutshell

  • you will need a license for Hadoop Analytics
  • You either use the metastore capability or you tell Splunk what datatype each Hive field
  • You tell Splunk the database and table name for Hive
  • You tell Splunk the path to the Hive data and what the db paths will look like
  • Splunk will run MUCH faster if your data has partitions

- setting up the provider can be a little bewildering if you have never done it

0 Karma

sabburisplunk
New Member

Thanks a lot. will try this. Just want to make sure, the splunk data here is not archived to Hadoop. We can directly map from Hive to Splunk data model.

0 Karma

burwell
SplunkTrust
SplunkTrust

Yes you associate a virtual index with a Hive table.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...