All Apps and Add-ons

Hadoop Monitoring: Where is

New Member

On the Hadoop Monitoring add-on details, it describes the inputs.conf entires for Spark. However, i didn't find it in /bin directory. Anyone has any insight into this please? Thx!

0 Karma

Splunk Employee
Splunk Employee

In the inputs.conf you can add spark monitoring, for example:

disabled = 0
sourcetype = hdfs_spark_metrics
index = spark_metricsdata

sourcetype = spark_historyserver
index = spark_metricsdata

sourcetype = spark_hivethrift
index = spark_metricsdata

And the in the you can add something like this

. dirname $0/
hadoop fs -text /spark-history/*

0 Karma
Get Updates on the Splunk Community!

Index This | Why do they call it hyper text?

November 2023 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

State of Splunk Careers 2023: Career Resilience and the Continued Value of Splunk

For the past three years, Splunk has partnered with Enterprise Strategy Group to conduct a survey that gauges ...

The Great Resilience Quest: 9th Leaderboard Update

The ninth leaderboard update (11.9-11.22) for The Great Resilience Quest is out >> Kudos to all the ...