Getting Data In

'Invalid Key in Stanza' errors being generated at startup for inputs.conf whitelist on a 6.1.4 Heavy Forwarder that docs say should work

wrangler2x
Motivator

Per these docs http://docs.splunk.com/Documentation/Splunk/6.1.4/Data/MonitorWindowsdata I have changed from the old way of using transforms to filter out unwanted Windows Events from logs I am monitoring to using a whitelist in inputs.conf. I am sending these to forwarders on various windows systems using deployment monitor. While restarting a Splunk forwarder that had died for some reason, I got this error on startup:

Invalid key in stanza [WinEventLog:Security] in C:\Program Files\Splunk\etc\apps\OIT_WINEVENT_DC_INDEX_WIN_01\default\inputs
.conf, line 23: whitelist (value: 528-535,539-540,624-626,632,636,659,642-644,660,675-676,671-672,680-681,1100,1102,1104,1108,4612,4616,4618,4624-4625,4634,4720,4728,4732,4738,4740,4756,4767-4768,4771-4772,4776,5461 )

Here is the stanza from the inputs.conf file in question:

[WinEventLog:Security]
disabled = 0
index= winevent_dc_index
whitelist = 528-535,539-540,624-626,632,636,659,642-644,660,675-676,671-672,680-681,1100,1102,1104,1108,4612,4616,4618,4624-4625,4634,4720,4728,4732,4738,4740,4756,4767-4768,4771-4772,4776,5461 

It looks just like the example in the documentation. Also, this blog entry says it should work in Splunk 6: http://blogs.splunk.com/2013/10/14/windows-event-logs-in-splunk-6/

So, why is this not working?

I also ran btool (it says the same thing):

C:\Program Files\Splunk\bin>splunk btool check --debug

Invalid key in stanza [WinEventLog:Security] in C:\Program Files\Splunk\etc\apps\OIT_WINEVENT_DC_INDEX_WIN_01\default\inputs.conf, line 23: whitelist  (value:  528-535,539-540,624-626,632,636,659,642-644,660,675-676,671-672,680-681,1100,1102,1104,1108,4612,4616,4618,4624-4625,4634,4720,4728,4732,4738,4740,4756,4767-4768,4771-4772,4776,5461 )
0 Karma

Michael_Carlisl
Explorer

So it seems that the issue is missing "\". If you update your inputs.conf file to be [WinEventLog://Security] it should work.

0 Karma

wrangler2x
Motivator

Wait, that's confusing... You say it is missing "\" then show "//" in [WinEventLog://Security]

But yeah, the docs at http://docs.splunk.com/Documentation/Splunk/6.0.2/Data/MonitorWindowsdata show it as "//".

I did not set up the stanzas we are using (which otherwise work fine without the "//") and the blog at http://blogs.splunk.com/2013/10/14/windows-event-logs-in-splunk-6/ shows it without the "//" so I did not event know these were missing until I read your comment. Did you test to see if adding them in allows the whitelist? Which version of Splunk?

0 Karma

Michael_Carlisl
Explorer

I tested it in version 6.3.1. Sorry about putting "\". There's also an issue with [WinEventLog:Application] if you ever use the Citrix Add Ons and use their inputs.conf file.

0 Karma

javiergn
SplunkTrust
SplunkTrust

Hi, your stanza looks all right to me.

  • Have you tried adding your event codes one at a time and see what happens?
  • Maybe the line is too long?
  • Or:

[MODE PARANOID ON]
Maybe the hyphen separating your event ID ranges is not the right type of hyphen.
See this: https://www.cs.tut.fi/~jkorpela/dashes.html

Or maybe there are hidden characters that your editor does not display
[MODE PARANOID OFF]

Hope that helps,
J

0 Karma

wrangler2x
Motivator

I removed all of the hyphens and used just a full list with commas only. Same error. Then I delected the whitelist line and manually added a new line which reads whitelist = 528 to keep it really simple, and so there was no chance of a hidden character. Same error on restart.

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi wrangler2x,

just a hint, look at the inputs.conf.spec file if this is listed

  whitelist = <regular expression>

Usually if you get this error something is missing in the .conf.spec file.

cheers, MuS

0 Karma

wrangler2x
Motivator

In the link you have there, there is a section called EventLog filtering which shows what the 6.1.4 documentation page I linked to (original question) describes. However, if I look at the actual 6.1.4 inputs.conf.spec I don't find that. Looks like my release may not support it, and the docs are wrong.

0 Karma

MuS
SplunkTrust
SplunkTrust

No, it more means someone forgot to add in the inputs.conf.spec and therefore splunk does not know about it and thinks it is an error.

BTW just downloaded a 6.1.4 Windows UF 64Bit and the input.conf.spec contains this on line 174:

 whitelist = <regular expression>

Maybe download your version again?

0 Karma

wrangler2x
Motivator

@MuS I have that same thing on line 174 also. But that is in the Monitor section. If you look at the latest documentation, under this section:

###
# Windows Event Log Monitor
###

You will find a sub-section which looks like this:

# EventLog filtering
#
# Filtering at the input layer is desirable to reduce the total processing load
# in network transfer and computation on the Splunk nodes acquiring and
# processing the data.

and in this section there is this:

whitelist = <list of eventIDs> | key=regex [key=regex]
blacklist = <list of eventIDs> | key=regex [key=regex]

And this:

The base unumbered whitelist and blacklist support two formats, a list of integer event IDs, and a list of key=regex pairs.

Now, in my 6.1.4 spec, in the Windows Event Log Monitor section, there is no subsection called EventLog filtering. However, just below the evt_dns_name = and index = specs, I do find these two (lines 1130 and 1141, respectively) :

whitelist = <list> | key=regex [key=regex]
blacklist = <list> | key=regex [key=regex]

And there are two comments with the whitelist which read:

 * In list form, tells Splunk which event IDs and/or event ID ranges that incoming events must have
  in order to be indexed.
 * In list form, A comma-separated list of event ID and event ID ranges to include (example: 4,5,7,100-200).

So, I'd say they are in the spec, although documented much differently than in the current documentation!

So, what next?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...