topic Filtering events not working in Getting Data In

Filtering events not working

remy06 — Mon, 09 Jan 2012 09:59:33 GMT

I've recently upgraded the forwarder to a universal forwarder on our app server.I'm collecting windows event logs as well as app logs on this server where previously parsing was done on it.

On the forwarder I've input for windows event logs and have configured to monitor the app log file. These are sent over to the indexer.Here's the configuration:

After the upgrade,I've moved the configuration to the indexer and it doesn't seem to work now.

app_log is the sourcetype for all logs collected from all app servers.I wish to filter the ip off on all app servers.
appserverA is the server that I wish to filter off eventcode 123,456,789,while the rest remain.

props.conf:

[app_log]  
TRANSFORMS-null = setnull 

 [host::appserverA]
TRANSFORMS-null = setnullappserverA

transforms.conf:

[setnull]
REGEX = (192\.168\.1\.10|192\.168\.1\.18)
DEST_KEY = queue
FORMAT = nullQueue

[setnullappserverA]
REGEX = EventCode=(123|456|789)
DEST_KEY = queue
FORMAT = nullQueue

An example of the event:

192.168.1.10 - - [09/Jan/2012:17:46:54] "message"   
192.168.1.18 - - [09/Jan/2012:17:46:55] "message"

Filtering for the eventcode is successful,but not for the IP.Can anyone advise if I've missed something?

Re: Filtering events not working

MuS — Mon, 09 Jan 2012 14:09:05 GMT

Hi remy06

I think your props.conf stanza [app_log] does not match your sourcetype; had this kind of problem before with some WMI sourcetypes.

Also try to simplify your regex for the IP to this one (\d+\.){3}(10|18)

cheers,
MuS

Re: Filtering events not working

remy06 — Mon, 28 Sep 2020 10:17:50 GMT

Hi MuS,
I've checked that [app_log] is the sourcetype.The sourcetype I've defined for application logs is app_log.Is there anything else I've missed?

Re: Filtering events not working

_d_ — Wed, 11 Jan 2012 03:51:02 GMT

What is the output of btool?

$SPLUNK_HOME/bin/splunk cmd btool --debug props list app_log

Re: Filtering events not working

remy06 — Wed, 11 Jan 2012 07:00:28 GMT

Here's the output

search     [app_log]
system     ANNOTATE_PUNCT = True
system     BREAK_ONLY_BEFORE =
system     BREAK_ONLY_BEFORE_DATE = True
system     CHARSET = UTF-8
system     DATETIME_CONFIG = /etc/datetime.xml
search     EXTRACT-Time = (?i)^[^"]*"(?P<Time>[^ ]*)(?= )
system     HEADER_MODE =
system     LEARN_SOURCETYPE = true
system     LINE_BREAKER_LOOKBEHIND = 100
system     MAX_DAYS_AGO = 2000
system     MAX_DAYS_HENCE = 2
system     MAX_DIFF_SECS_AGO = 3600
system     MAX_DIFF_SECS_HENCE = 604800
system     MAX_EVENTS = 256
system     MAX_TIMESTAMP_LOOKAHEAD = 128
system     MUST_BREAK_AFTER =
system     MUST_NOT_BREAK_AFTER =
system     MUST_NOT_BREAK_BEFORE =
system     SEGMENTATION = indexing
system     SEGMENTATION-all = full
system     SEGMENTATION-inner = inner
system     SEGMENTATION-outer = outer
system     SEGMENTATION-raw = none
system     SEGMENTATION-standard = standard
system     SHOULD_LINEMERGE = True
system     TRANSFORMS =
system     TRANSFORMS-null = setnull
system     TRUNCATE = 10000
system     maxDist = 100

Re: Filtering events not working

remy06 — Tue, 31 Jan 2012 10:03:44 GMT

any status?