Getting Data In

Filter data out from Data Input

bahmed
New Member

Hi,

I have spent last 2 hours searching for this simple scenario on Splunk Answers, without any luck.
Here is the case.

  • Splunk 6.0.2 (Trial version)
  • OS : Windows 7, 64 Bit
  • Data Input : A Log4J file on my local computer

Requirement : Just want to index events which contains the string "[ERROR ]", in my indexer.

Any help will be greatly appreciated.

Tags (2)
0 Karma

yannK
Splunk Employee
Splunk Employee

Setup those type of filters at the indexer level :
You can use a rule based on the sourcetype, and a matching regex based on the event.
You can test your regex on sample events with the "rex" command in splunk before to make sure.

http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Keep_specific_even...

bahmed
New Member

For the new events, all of them are getting indexed including the one contains "Error".

I have restarted the Splunk on my local machine.

0 Karma

yannK
Splunk Employee
Splunk Employee

This looked correct, What is the current behavior when you index new events :

  • all events are dropped the nullQueue (with or without the [ERROR] keyword)
  • all events are indexed ?
  • a mix of both ?

did you restarted the indexers to apply the change ?

0 Karma

bahmed
New Member

Thanks yannK. I have changed the regular expression as per your point 1. No change in the result.

My props/transforms are in the following directory : C:\Program Files\Splunk\etc\system\local

Does that makes them on indexers.

Appreciate your help.

0 Karma

yannK
Splunk Employee
Splunk Employee

1- [ and ] are regex keyword, you should escape them
REGEX = \[ERROR\]

2- make sure that those props/transforms are on the indexers (not on universal or lightweight forwarders only)

0 Karma

bahmed
New Member

Based on the article, I set the following files as shown, but still not getting the filtered log.

props.config

[MyVacationLog]
TRANSFORMS-set= myvacnull,myvacparsing

transforms.config

[myvacnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[myvacparsing]
REGEX = [ERROR]
DEST_KEY = queue
FORMAT = indexQueue

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...