topic Re: Filter strings of event before index in Getting Data In

Filter strings of event before index

thinksplunk — Tue, 24 Sep 2013 13:53:20 GMT

Hi, as i'm new to using Splunk, i would like to know how to filter the string "2013-09-20 16:53:00, 231 Success transport" before it is index, how to do it and where to configure it? thks

2013-09-20 16:53:04,723 INFO[Thread-3]EndTime=20/09/2013 16:53:04 TransactionID=A, Event=completed, Result=sent

2013-09-20 16:53:00, 231 Success transport

2013-09-20 16:52:04,723 INFO[Thread-3]StartTime=20/09/2013 16:52:04 TransactionID=A, Event=start_process

Re: Filter strings of event before index

kristian_kolb — Tue, 24 Sep 2013 14:07:43 GMT

You can filter out events before they are indexed with the help of a so-called nullQueue transform. Please note that the regex below is pretty specific in order not to remove events you want to keep.

props.conf

[your_source_or_sourcetype]
TRANSFORM-blah = remove_success_transport

transforms.conf

[remove_success_transport]
REGEX = ^\d+-\d+-\d+\s\d+:\d+:\d+,\d+\sSuccess\stransport$
DEST_KEY = queue
FORMAT = nullQueue

For more info;

http://docs.splunk.com/Documentation/Splunk/5.0.5/Deploy/Routeandfilterdatad#Discard_specific_events_and_keep_the_rest

Re: Filter strings of event before index

thinksplunk — Tue, 01 Oct 2013 02:41:27 GMT

To be exact, i need to filter the string is "2013-09-20 16:53:34,374 INFO [Thread-2] [null:-1] Success. Connected to AB Server at 192.11.12.13:123.

Re: Filter strings of event before index

kristian_kolb — Tue, 01 Oct 2013 05:26:03 GMT

Did you read the linked doc page? Given that you understand the basic principle of how routing events to the nullQueue works, it's just a question of writing the right regex.

Re: Filter strings of event before index

thinksplunk — Tue, 01 Oct 2013 06:05:09 GMT

correct, it is writing the right regex, as i mentioned i'm new in using Splunk, so not very sure how to write the right regex.

Re: Filter strings of event before index

JimDeich — Wed, 05 Nov 2014 02:53:38 GMT

It seems like the poster was just looking to remove PART of a log or event, but the answer given will remove the ENTIRE EVENT .