Getting Data In

Transform/props not working.

mileven
Explorer

I am trying to filter out Windows Event logs and only allow Errors and Critical event logs to be indexed and I want to drop everything else.

Props.conf
[WinEventLog:Application]
TRANSFORMS-FilterEvents = FilterWarningEvents, FilterInformationEvents
[WinEventLog:Security]
TRANSFORMS-FilterEvents = FilterWarningEvents, FilterInformationEvents
[WinEventLog:System]
TRANSFORMS-FilterEvents = FilterWarningEvents, FilterInformationEvents

Transform.conf
[FilterInformationEvents]
REGEX = (?ms)(Type=Information)
DEST_KEY = queue
FORMAT = nullQueue

[FilterWarningEvents]
REGEX = (?ms)(Type=Warning)
DEST_KEY = queue
FORMAT = nullQueue

These are on my indexer but I still see informational and warnings being indexed.

0 Karma

yannK
Splunk Employee
Splunk Employee

To verify that your regex actually match your events in splunk. Try to search in splunk with the regex command :

sourcetype=WinEventLog:Application OR sourcetype=WinEventLog:System OR sourcetype=WinEventLog:Security | regex _raw="(?ms)(Type=Information)" | table Type sourcetype _raw

maybe are you not accounting for some spaces or separators.

rsennett_splunk
Splunk Employee
Splunk Employee

Those look like their coming from Snare, which strips the multiline windows logs into a single line syslog style log where the key value pairs are delimited with an equal sign.

typically the sourcetype for that is windows_snare_syslog. I'm wondering if you are confusing sourcetypes and if those sourcetypes you've listed are actually pointing at the traditional multiline events that Ayn has described...

Ayn's reference to the spelling of transform(S) was because you left the plural S off of the word in your example... the file must be named with the plural to work...

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

saurabh_tek11
Communicator

Here is just an example of event ID 1500 - where it says

Event ID: 1500
Task Category: None
Level: Error

this signifies there is some alteration in logs while coming from source as @rsennett suggested

0 Karma

mileven
Explorer

LogName=System SourceName=Microsoft-Windows-GroupPolicy EventCode=1500 EventType=4 Type=Information ComputerName=STRR1INFHPV01.redmond.corp.microsoft.com User=SYSTEM Sid=S-1-5-18 SidType=1 TaskCategory=None OpCode=Start RecordNumber=46013 Keywords=None

pulled directly from what is being indexed.

0 Karma

Ayn
Legend

OK, as long as you're not confusing the field "Type" in Splunk having the value "Information" with that the raw event actually has the exact string "Type=Information" in it. Maybe a good idea to paste a sample event here to make sure this is not the case.

0 Karma

mileven
Explorer

Type is Type=Information. Not sure what you mean by make sure you file is called in transforms.conf

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Make sure your file is called transform*s*.conf.

Ayn
Legend

Do your events have these exact strings in them, i.e. "Type=Information"? Usually Windows logs rather have something like "Type: Information" instead.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...