do not index events using props/transforms regex help

[mjm295]

Hi

A recent agent install across our infrastructure has created a flood in the proxy logs of blocked messages which is blowing out our license.

Until there is a proper fix I need to stop ingesting events related to 1 URL.

I think  I just need help with the REGEX part.

Log are dropped to a syslog server running a heavy forwarder, then we run a monitor on that log file.

Inputs.conf

[monitor:///remotesyslogs/mgmt-austaiaecho00*/*.log]
disabled = false
index = star_proxy
sourcetype = cisco:wsa:squid

props.conf

[cisco:wsa:squid]
TRANSFORMS-screen=eliminate-screenconnect

transforms.conf

[eliminate-screenconnect]
REGEX = ?.=screenconnect
DEST_KEYi = queue
FORMAT = nullQueue

Example event from log file:

2020-06-12T04:04:55+10:00 mgmt-austaiaecho005.casino.internal accesslogs_splunk: Info: 1591898695.320 0 10.10.216.100 TCP_DENIED/407 0 CONNECT tunnel://screenconnect.techmedia.com.au:8080/ - NONE/- - OTHER-NONE-DefaultGroup-NONE-NONE-NONE-NONE <-,-,-,"-",-,-,-,-,"-",-,-,-,"-",-,-,"-","-",-,-,-,-,"-","-","-","-","-","-",0.00,0,-,"-","-",-,"-",-,-,"-","-",-,-,"-"> - - "12/Jun/2020:04:04:55 +1000" -

Basically for next 4 weeks I need to drop all events with

 screenconnect.techmedia.com.au

Thanks

Mark

[to4kawa]

[eliminate-screenconnect]
REGEX = screenconnect\.techmedia\.com\.au
DEST_KEY = queue
FORMAT = nullQueue

A simple REGEX is enough for nullQueue.

[alonsocaio]

Hi,

I would try to use some regex like this one:

^.+screenconnect\.techmedia\.com\.au.+$

I have tested it using Regex101 (https://regex101.com/r/gHJVDp/2), with the URLs below:

screenconnect.techmedia.com.au
screenconnect2.techmedia.com.au
screenconnect.techmedia10.com.au

* I also recommend you to test this regex (and any other you build or find) in a dev environment, before using it in your production transforms.conf file.