topic Re: Invalid Key In Stanza - index and sourcetype in Getting Data In

Invalid Key In Stanza - index and sourcetype

StuartMacL — Thu, 10 Aug 2023 08:16:49 GMT

On my deployment server, when running btool check against inputs.conf and 'grep'ing for the name of my manually created app (which has nothing but a local directory, an inputs.conf and an automatically created app.conf file) I have a 'Invalid key in stanza [monitor]...' which complains about a line where I have;
index = indexName

And another error about;
sourcetype = sourcetypeName

I don't understand why Splunk doesn't like these lines. I can't find an appropriate inputs.conf.spec file where the issue could be fixed, but maybe I am not looking in correct place.

When I run a btool check against all of our .conf files and Splunk is reporting that fields such as index, source, sourcetype, crcSalt, initCrcLength and more are invalid stanzas.

We have hundreds of such ‘invalid key’ errors. We also have hundreds of errors for “No spec file for:” for all .conf files other than inputs.conf – no such errors for inputs.conf. Maybe something major (or minor with major implications) went wrong after an upgrade?

Re: Invalid Key In Stanza - index and sourcetype

gcusello — Thu, 10 Aug 2023 08:42:54 GMT

Hi @StuartMacL,

could you share your inputs.conf?

where is it located (full path)?

with btool you analyze installed apps, but on the deployment server, usually there isn't any installed apps, but only apps in $SPLUNK_HOME/etc/deployment-apps to deploy.

If you run btool on a target client, have you the same message?

Ciao.

Giuseppe

Re: Invalid Key In Stanza - index and sourcetype

StuartMacL — Thu, 10 Aug 2023 09:13:35 GMT

Hi gcusello, appreciate your speedy response.

This is the full inputs.conf;

[monitor:///var/log/beams/.*log]
disabled = 0
index = beams
sourcetype = gobeams

It is located in /opt/splunk/etc/deployment-apps/TA-GOBEAMS-inputs/local/inputs.conf

Yes, I had to copy all my apps from deployment-apps directory into a new directory named deployment-apps-for-btool/apps/ and then specify that btool check that directory.

Regarding running btool on server where UF is, I cant do that currently but have asked manager of server to do so.

I'm wondering if there is some bigger issue as many of our apps/feeds have stopped working and I also see many Invalid stanza errors, for GOBEAMS app but also other apps where events are no longer being received by our indexers.

Re: Invalid Key In Stanza - index and sourcetype

gcusello — Thu, 10 Aug 2023 15:20:16 GMT

Hi @StuartMacL,

if you cannot access the target client, could you install the TA on a different client (eventually a test system)?

Could you access the target client only to run a local Splunk restart by console and see the boot messages?

As I said the btool should be applied on installed apps and probably the issue is this.

Ciao.

Giuseppe

Re: Invalid Key In Stanza - index and sourcetype

StuartMacL — Thu, 10 Aug 2023 15:50:52 GMT

I deployed the app on my local machine and ran btool - i think you are correct, i dont get the same invalid stanza errors there.

I wish there was some way to check for issues with conf files in deployment-apps. Is the only way to do this by copying your app to apps directory?

Re: Invalid Key In Stanza - index and sourcetype

gcusello — Thu, 10 Aug 2023 15:57:19 GMT

Hi @StuartMacL,

I didn't used it in deployment-apps folder, so I'm not sure that's possible on deployment-apps.

Ciao.

Giuseppe

Re: Invalid Key In Stanza - index and sourcetype

isoutamo — Fri, 11 Aug 2023 10:41:07 GMT

basically you could do the check with btool like this

[soutamo@vega] ~/tmp> (0) $ splunk btool --dir=`pwd`/etc check Invalid key in stanza [monitor:///var/log/com.apple.xpc.launchd/launchd.log] in /Users/soutamo/tmp/etc/apps/_server_app_Launchd/local/inputs.conf, line 4: foo (value: bar).

But you need to copy also etc/system/README folder under your etc to get it working.

r. Ismo