Getting Data In

My scripted input is working, but why is it assigning an incorrect hostname for 1 of 6 target hosts?

pipegrep
Path Finder

I have a simple scripted input that runs a powermt command periodically.

<apps>/powermt/bin/powermt.sh

#!/bin/bash
sudo powermt display

<apps>/powermt/local/inputs.conf

##### Powermount scripted Inputs ######1
[script:///opt/splunkforwarder/etc/apps/powermt/bin/powermt.sh]
## Run every 15 minutes
disabled = false
interval = 15
source = powermnt
sourcetype = script:///opt/splunkforwarder/etc/apps/powermt/bin/powermt.sh]

<apps>/local/props.conf

[powermt]
DATETIME_CONFIG = CURRENT

The app deploys and runs as it should on 5 out of the 6 target hosts otherwise , however on the 6th host it attributes the data to an incorrect host name (a test server that has been retired).

The good news is that there's lots documentation about this. The bad news is that there's lots of documentation about this. While I go back and convert several hours of my day into unraveling the root issue, I'm wondering if there's a way I can force the script to force/set the local hostname. This would relieve quite a bit of my immediate pressure.

0 Karma
1 Solution

acharlieh
Influencer

In inputs.conf there is a host attribute that you could set on the script stanza to seed the appropriate value (and assuming your indexer isn't overriding this value at parse time, this is what your events will be set to). Let's take a peek at the resolved config, so in your splunk home, run: bin/splunk btool inputs list script --debug

This will spit out the resolved configuration of all your script stanzas (for a global time configuration as inputs is), and the path to the inputs.conf file that each line is resolved from. You may see a host already being set for the stanza in question (likely in etc/system/local/inputs.conf ).

If you're curious about this resolution of files you may be interested in docs about how this resolution works or even more documentation about btool, or a video on how to “Support Splunk like A Support Professional – Episode 3 – Troubleshooting Configuration Issues”.

Now unrelated to the host problem, you have one other issue, that your props.conf is configuring a stanza to point to the sourcetype of powermt, but your stanza is setting the source to powermt.

View solution in original post

acharlieh
Influencer

In inputs.conf there is a host attribute that you could set on the script stanza to seed the appropriate value (and assuming your indexer isn't overriding this value at parse time, this is what your events will be set to). Let's take a peek at the resolved config, so in your splunk home, run: bin/splunk btool inputs list script --debug

This will spit out the resolved configuration of all your script stanzas (for a global time configuration as inputs is), and the path to the inputs.conf file that each line is resolved from. You may see a host already being set for the stanza in question (likely in etc/system/local/inputs.conf ).

If you're curious about this resolution of files you may be interested in docs about how this resolution works or even more documentation about btool, or a video on how to “Support Splunk like A Support Professional – Episode 3 – Troubleshooting Configuration Issues”.

Now unrelated to the host problem, you have one other issue, that your props.conf is configuring a stanza to point to the sourcetype of powermt, but your stanza is setting the source to powermt.

pipegrep
Path Finder

Thank you. Yes, btool is my friend.

Sure enough;
/opt/splunkforwarder/etc/system/local/inputs.conf had a server name forced into it. I have no idea why, though that probably merits some review. In the meantime I'll clear out the old data and let this start reindexing as it should.

0 Karma

acharlieh
Influencer

It's been my observation that, Splunk (currently, at least since 6.0) sets a host name into that file on first run. What I suspect happened, is that the Splunk UF was setup once on one box, and then the directory was cloned between machines or a source machine was imaged with it. I think there's instructions for steps to follow to install a UF as part of a system image somewhere, but I'd have to go digging for a link.

0 Karma

pipegrep
Path Finder

This was a physical system that had been repurposed to a degree.I bet you're right.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...