All Apps and Add-ons

Hadoop Monitoring: Where is hadoopmon_spark.sh?

baloney
New Member

On the Hadoop Monitoring add-on details, it describes the inputs.conf entires for Spark. However, i didn't find it in /bin directory. Anyone has any insight into this please? Thx!

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

In the inputs.conf you can add spark monitoring, for example:

[script://./bin/hadoopmon_spark.sh]
disabled = 0
interval=60
sourcetype = hdfs_spark_metrics
index = spark_metricsdata

[monitor:///var/log/spark/history]
sourcetype = spark_historyserver
index = spark_metricsdata

[monitor:///var/log/spark/thriftserver]
sourcetype = spark_hivethrift
index = spark_metricsdata

And the in the hadoopmon_spark.sh you can add something like this

. dirname $0/hadoopmon_common.sh
hadoop fs -text /spark-history/*

0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...