I'm using splunk enterprise on a local windows based system.
I have a file reader configured to watch a directory where I dump logs and folders of logs.
c:\logs\*\*.log
All folders and files that end in ".log"
There is a specific event that is typically in my .log files and they always start with 30 and 32. I'd like to filter this out and I've tried everything I can think of.
I even copied this type of setup, but I can't seem to get it working:
Section: "Discard specific events and keep the rest"
http://docs.splunk.com/Documentation/Splunk/6.2.2/Forwarding/Routeandfilterdatad
Used this for a reference for windows file paths:
http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Specifyinputpathswithwildcards
in the etc\system\local
props.conf
[source::....log]
TRANSFORMS-null= setnull
Also tried [source:://....log]
Also tried [monitor:://....log]
Also tried [monitor::....log]
transforms.conf
[setnull]
regex = ^3[02]
DEST_KEY = queue
FORMAT = nullQueue
After making changes, I restart splunk and send some test data, every time, my unwanted events that start with 30 and 32 still show up. Any help would be great, I'm pretty sure my regex is right, but I don't have any idea if the rest is.
Thanks,
Grant
First remark, do not use "setnull" as a transforms name, it's too generic ,and could overwrite an existing definition.
Prefer something that describe better like : "setnull_logfilter"
Second remark, maybe a typo
TRANSFROMS-null= setnull
should be
TRANSFORMS-null= setnull
tried:
TRANSFORMS-null = setnull_dhcp
Also didn't work, I did change the transforms.conf file too when doing this name change.
The next step is to figure is you have a single instance or if this forwarder is sending data to another instance (indexer, or heavy forwarder)
The index time rules have to be setup on the instance that is parsing the events : the indexers (or the intermediary heavy forwarder if any)
example of forwarding architectures :
UF -> IDX (put rules here)
UF -> UF -> IDX (put rules here)
UF -> HF (put rules here) -> IDX
UF -> IDX (put rules here) -> IDX
IDX (put rules here)
HF (put rules here) -> IDX
it's basically just a single instance test box, not forwarding any data.
Ok. so let's try with a broader props.conf condition
[source::*log]
I found the input type by looking at etc\apps\search\local\inputs.conf
Turns out it's [monitor://C:\Logs\dhcplogs]
Tried both with
TRANSFORMS-null = setnull_dhcp
Still not filtering correctly. Is my regex wrong? do I need to stick this into a different .conf file?
How can I find out the correct type for the [source] or [monitor]?
Yes, typo sorry.