Getting Data In

do not index events using props/transforms regex help

mjm295
Path Finder

Hi

A recent agent install across our infrastructure has created a flood in the proxy logs of blocked messages which is blowing out our license.

Until there is a proper fix I need to stop ingesting events related to 1 URL.

I think  I just need help with the REGEX part.

Log are dropped to a syslog server running a heavy forwarder, then we run a monitor on that log file.

Inputs.conf

 

[monitor:///remotesyslogs/mgmt-austaiaecho00*/*.log]
disabled = false
index = star_proxy
sourcetype = cisco:wsa:squid

 

 

props.conf

 

[cisco:wsa:squid]
TRANSFORMS-screen=eliminate-screenconnect

 

 

transforms.conf

 

[eliminate-screenconnect]
REGEX = ?.=screenconnect
DEST_KEYi = queue
FORMAT = nullQueue

 

 

Example event from log file:

 

2020-06-12T04:04:55+10:00 mgmt-austaiaecho005.casino.internal accesslogs_splunk: Info: 1591898695.320 0 10.10.216.100 TCP_DENIED/407 0 CONNECT tunnel://screenconnect.techmedia.com.au:8080/ - NONE/- - OTHER-NONE-DefaultGroup-NONE-NONE-NONE-NONE <-,-,-,"-",-,-,-,-,"-",-,-,-,"-",-,-,"-","-",-,-,-,-,"-","-","-","-","-","-",0.00,0,-,"-","-",-,"-",-,-,"-","-",-,-,"-"> - - "12/Jun/2020:04:04:55 +1000" -

 

 

Basically for next 4 weeks I need to drop all events with

 

 screenconnect.techmedia.com.au

 

 

 

Thanks

Mark

 

 

Labels (1)
0 Karma

to4kawa
SplunkTrust
SplunkTrust
[eliminate-screenconnect]
REGEX = screenconnect\.techmedia\.com\.au
DEST_KEY = queue
FORMAT = nullQueue

A simple REGEX is enough for nullQueue.

0 Karma

alonsocaio
Contributor

Hi,

I would try to use some regex like this one:

 

^.+screenconnect\.techmedia\.com\.au.+$

 

I have tested it using Regex101 (https://regex101.com/r/gHJVDp/2), with the URLs below:

 

screenconnect.techmedia.com.au
screenconnect2.techmedia.com.au
screenconnect.techmedia10.com.au

 

* I also recommend you to test this regex (and any other you build or find) in a dev environment, before using it in your production transforms.conf file.

Tags (1)
0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!