Getting Data In

Blacklist using props.conf and transforms.conf

splunkcol
Contributor

I need to reject or not index the logs that have the word "notice" inside the log

I understand that it is done using these two files

I have 2 doubts:

1. Is the regex ok?
2. If the path is constantly changing I can use a wildcard? [source::/folder/folder/logs/firewall-xxxxx/* ]

props.conf

[source::/folder/folder/logs/firewall-xxxxx/2020/12/4/local7.log]
TRANSFORMS-null= setnull

transforms.conf

[setnull]
REGEX = notice
DEST_KEY = queue
FORMAT = nullQueue

Sample Log

date=2019-05-10 time=11:37:47 logid="0000000013" type="traffic" subtype="forward" level="notice" vd="vdom1" eventtime=1557513467369913239 srcip=10.1.100.11 srcport=58012 srcintf="port12" srcintfrole="undefined" dstip=23.59.154.35 dstport=80 dstintf="port11" dstintfrole="undefined" srcuuid="ae28f494-5735-51e9-f247-d1d2ce663f4b" dstuuid="ae28f494-5735-51e9-f247-d1d2ce663f4b" poluuid="ccb269e0-5735-51e9-a218-a397dd08b7eb" sessionid=105048 proto=6 action="close" policyid=1 policytype="policy" service="HTTP" dstcountry="Canada" srccountry="Reserved" trandisp="snat" transip=172.16.200.2 transport=58012 appid=34050 app="HTTP.BROWSER_Firefox" appcat="Web.Client" apprisk="elevated" applist="g-default" duration=116 sentbyte=1188 rcvdbyte=1224 sentpkt=17 rcvdpkt=16 utmaction="allow" countapp=1 osname="Ubuntu" mastersrcmac="a2:e9:00:ec:40:01" srcmac="a2:e9:00:ec:40:01" srcserver=0 utmref=65500-742

 
 

splunkcol_0-1607090305117.png

 

0 Karma
1 Solution

twinspop
Influencer

The regex is simple. You'd have to answer if it's appropriate or not. Any log event with "notice" anywhere in the event will match.  

Pattern matching in the `[source::]` qualifier works like it does with inputs. `*` matches anything but file delimiters, and `...` matches anything. Something like this might work   

`[source::/folder/folder/logs/firewall-*/*/*/*/local*.log]`

View solution in original post

splunkcol
Contributor

If someone is helpful, this only applies to Heavy forwarders

In Universal forwarder there is no filtering capability through regular expressions

0 Karma

twinspop
Influencer

The regex is simple. You'd have to answer if it's appropriate or not. Any log event with "notice" anywhere in the event will match.  

Pattern matching in the `[source::]` qualifier works like it does with inputs. `*` matches anything but file delimiters, and `...` matches anything. Something like this might work   

`[source::/folder/folder/logs/firewall-*/*/*/*/local*.log]`

View solution in original post

.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!