Splunk Search

what could account for changes in case for input sources?

dstaulcu
Builder

When constructing a search to render a table of count of events by source I noticed that splunk was treating the identical input sources as different based upon differences in their source name character case. For example, one result rendered WindowsEvent:Application and WindowsEvent:application as different source types.

I reviewed inputs.conf for hosts associated with WindowsEvent:application events and verified that their their input definitions did not contain a lower-case instance of "A" in source name "Application".

Can anyone think of anything that could account for such a change anywhere along the data processing path between universal forwarders and search heads? -our path includes UF --> HF --> IDX --> SH.

I've been working around this problem by augmenting searches with a search time function of | eval source=lower(source) | stats count(host) by source... but, I'm worried that other folks might know about this issue and thus not incorporate such workarounds so I'd like to correct the problem at the source. -pun intended

All components of our splunk implementation (universal forwarders and servers) are >= 6.0

Masa
Splunk Employee
Splunk Employee

If you're experiencing this issue with Windows 2k3 server, it is a known issue;

http://answers.splunk.com/answers/118453/wineventlog-filters-failing-windows-2003-and-splunk-6-spl-7...

http://docs.splunk.com/Documentation/Splunk/latest/ReleaseNotes/Knownissues
On Windows Server 2003, the WinEventLog input generates sourcetypes in all lower case, for example, WinEventLog:security versus WinEventLog:Security. This can cause filters that have been set up in props.conf to not match, which can ultimately result in unexpected indexing of data. To work around the problem, follow the instructions in "Windows Event Log filters fail" on Splunk Answers. (SPL-78726)

dstaulcu
Builder

that search didn't reveal anything. I did a full text search of files in installdir on splunk server and don't see any references of concern to the lower case version of the source. I did the same search on a universal forwarder (v6) from which events were logged and while I don't see references to lower case instance of source name in conf files I do see references to it in metrics.log. I wonder if the problem has anything to do with new features to whitelist or blacklist events from forwarding...

0 Karma

lukejadamec
Super Champion

I get bored easy.
I can't test it, and I think I had it backward, tho it should still work...
You can have many sources for a single sourcetype, and you should include a trailing * in the source search.
index=main source="*application*" |dedup sourcetype| table source,sourcetype

0 Karma

dstaulcu
Builder

dang dude.. you're everywhere in community postings.. thanks for the input.. I'll check that out!

lukejadamec
Super Champion

Check the WindowsEvent:application sourcetypes:
index=main source="*application" |dedup sourcetype| table source,sourcetype
If you find that the lower case application is restricted to particular sourcetypes, then check props.conf and transforms.conf on the indexer and forwarder for all apps that process windows events.

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...