Getting Data In
Highlighted

Filtering events using NullQueue

Contributor

I was wondering if there is any way to filter eventcodes, but not every event that is being passed through. For example is there a way to block EventCode 4624, but just the debug messages and let the rest pass?

This is what we currently have to block windows EventCodes:

REGEX=EventCode=(4624|4776|4662|4634|4688|4648|4907|4768|4624||538|560|552|534)

We want to remove EventCode=4624 leaving the rest. the EventCode=4624 is generated because of "An account was successfully logged on" event on all servers. We want this enabled for most windows servers, but want to block this event from our 13 domain controller hostnames. Is it possible to have multiple regexes sending to null Queues?

Highlighted

Re: Filtering events using NullQueue

Legend

To use the nullQueue, you must be able to write a regular expression that identifies the events to be eliminated.

For the event code, that would be

EventCode\s*=\s*4624

but I am not sure how you would identify this as a debug message. Can you post an example of a few events?

Highlighted

Re: Filtering events using NullQueue

Contributor

I dont know if that is exactly what i was looking for. I probably worded the question in a confusing way.

Heres another example:
we want to remove EventCode=4624 leaving the rest. the EventCode=4624 is generated because of "An account was successfully logged on" event on all servers. We want this enabled for most windows servers, but want to block this event from our 13 domain controller hostnames. Is it possible to have multiple regexes sending to null Queues? AND if so how would we do this?

0 Karma
Highlighted

Re: Filtering events using NullQueue

Ultra Champion

Yes you can have multiple transforms that send stuff to the null queue;

props.conf
[sourcetypex]
TRANSFORMS-delete
stuff = dropa, dropb

transforms.conf
[drop_a]
REGEX = a
DEST = queue
FORMAT = nullQueue

[drop_b]
REGEX = b
DEST = queue
FORMAT = nullQueue

That's the same as having REGEX = a|b in one nullQueue transform.

0 Karma
Highlighted

Re: Filtering events using NullQueue

Communicator

What would my REGEX line in the transforms.conf be to ELIMINATE any events that don't have this string? I must be missing something. I only want to ingest events that have this string at the beginning of the line:  "|>>>>>>|" 

In REGEX that should be ^\|>>>>>>\| right? 

So how to i set the transforms.conf REGEX= line to say anything that doesn't match the above REGEX, drop to the nullqueue?

 

Thanks in advance!!!

Joe

0 Karma
Highlighted

Re: Filtering events using NullQueue

Ultra Champion

If you by debug mean the Type=Debug (I don't know if it exists, I only have 'Informational' in my logs). Therefore I used ComputerName in the example below.

The following regex works with rex inline in the search. It should probably work with the instructions in http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Discardspecificeventsandkeeptherest

sourcetype=wineventlog:security EventCode="4624" | rex ".*(?<blaha>EventCode=4624[\n\r\w\.=]*ComputerName=some.host.name).*"

Hope this helps,

Kristian

Highlighted

Re: Filtering events using NullQueue

Contributor

Thank you for your help Kristian.

0 Karma
Highlighted

Re: Filtering events using NullQueue

Contributor

This is what we currently have to block windows EventCodes:

REGEX=EventCode=(4624|4776|4662|4634|4688|4648|4907|4768|4624||538|560|552|534)

We want to remove EventCode=4624 leaving the rest. the EventCode=4624 is generated because of "An account was successfully logged on" event on all servers. We want this enabled for most windows servers, but want to block this event from our 13 domain controller hostnames. Is it possible to have multiple regexes sending to null Queues?

0 Karma
Highlighted

Re: Filtering events using NullQueue

Legend

[Updated to show that you can do multiple transforms]

Okay - given the answer from Kristian about the type, I think I can show you how to filter the events. Assuming that the sourcetype is called WinEventLog:Security...

props.conf

[WinEventLog:Security]
TRANSFORMS-t1=eliminate-4624-debug
TRANSFORMS-t2=eliminate-eventcodes

transforms.conf

[eliminate-4624-debug]
REGEX=(?m)EventCode\s*=\s*4624.*?Type\s*=\s*Debug\s
DEST_KEY=queue
FORMAT=nullQueue

[eliminate-eventcodes]
REGEX=EventCode=(4776|4662|4634|4688|4648|4907|4768|4624||538|560|552|534)
DEST_KEY=queue
FORMAT=nullQueue

Now, this is not the tightest regular expression, so I would test it with the following search:

sourcetype="WinEventLog:Security" 
| regex _raw="(?m)EventCode\s*=\s*4624.*?Type\s*=\s*Debug\s"

If this search matches only the data that you want to eliminate, then great. Otherwise, I may still need to see a sample of the data...

View solution in original post

Highlighted

Re: Filtering events using NullQueue

Ultra Champion

well, I made an assumption regarding the Type=Debug... I'd also like sample data...

/k

0 Karma