I've recently upgraded the forwarder to a universal forwarder on our app server.I'm collecting windows event logs as well as app logs on this server where previously parsing was done on it.
On the forwarder I've input for windows event logs and have configured to monitor the app log file. These are sent over to the indexer.Here's the configuration:
After the upgrade,I've moved the configuration to the indexer and it doesn't seem to work now.
app_log is the sourcetype for all logs collected from all app servers.I wish to filter the ip off on all app servers.
appserverA is the server that I wish to filter off eventcode 123,456,789,while the rest remain.
[app_log] TRANSFORMS-null = setnull [host::appserverA] TRANSFORMS-null = setnullappserverA
[setnull] REGEX = (192\.168\.1\.10|192\.168\.1\.18) DEST_KEY = queue FORMAT = nullQueue [setnullappserverA] REGEX = EventCode=(123|456|789) DEST_KEY = queue FORMAT = nullQueue
An example of the event:
192.168.1.10 - - [09/Jan/2012:17:46:54] "message" 192.168.1.18 - - [09/Jan/2012:17:46:55] "message"
Filtering for the eventcode is successful,but not for the IP.Can anyone advise if I've missed something?
Here's the output
search [app_log] system ANNOTATE_PUNCT = True system BREAK_ONLY_BEFORE = system BREAK_ONLY_BEFORE_DATE = True system CHARSET = UTF-8 system DATETIME_CONFIG = /etc/datetime.xml search EXTRACT-Time = (?i)^[^"]*"(?P<Time>[^ ]*)(?= ) system HEADER_MODE = system LEARN_SOURCETYPE = true system LINE_BREAKER_LOOKBEHIND = 100 system MAX_DAYS_AGO = 2000 system MAX_DAYS_HENCE = 2 system MAX_DIFF_SECS_AGO = 3600 system MAX_DIFF_SECS_HENCE = 604800 system MAX_EVENTS = 256 system MAX_TIMESTAMP_LOOKAHEAD = 128 system MUST_BREAK_AFTER = system MUST_NOT_BREAK_AFTER = system MUST_NOT_BREAK_BEFORE = system SEGMENTATION = indexing system SEGMENTATION-all = full system SEGMENTATION-inner = inner system SEGMENTATION-outer = outer system SEGMENTATION-raw = none system SEGMENTATION-standard = standard system SHOULD_LINEMERGE = True system TRANSFORMS = system TRANSFORMS-null = setnull system TRUNCATE = 10000 system maxDist = 100
I think your props.conf stanza
[app_log] does not match your sourcetype; had this kind of problem before with some WMI sourcetypes.
Also try to simplify your regex for the IP to this one