Knowledge Management

Searching data from hadoop data roll on HDFS with Hive?

driekhof
Path Finder

We use the Splunk Hadoop Data Roll to move our frozen data over to our Hadoop cluster.  The writing of the data to HDFS seems to work pretty well, but the searching of it through Splunk doesn't work well at all.  We get lots of different errors from the query not parsing correctly (some problem with how splunk translates the parenthesis) or some mysterious error happens in the MR job on Hadoop.

We use Cloudera, and would like to be able to query the data there through Hue/Hive as an alternative to our terrible experience trying to query the hadoop data through Splunk.   Can anyone offer guidance on how to query the 'rolled' data on a Cloudera Hadoop cluster without going through Splunk search?

 

Labels (1)
0 Karma

driekhof
Path Finder

Forgot to mention, another really annoying error:  Splunk frequently submits jobs to our standby resource manager instead of the active one even though we've configured the HA stuff in Splunk.

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Enterprise 9.4: Features to Power Your Digital Resilience

Hey Splunky People! We are excited to share the latest updates in Splunk Enterprise 9.4. In this release we ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...