Splunk Search

How to filter out syslog events from a group of hosts via regex?

AKG
Path Finder

Hi

We have a group of servers and looks like they have been reconfigured. Until we get hold of a sysadmin and fix the issue, we have to drop off logs from those servers.

Below is how we tried filtering with no success.

props.conf

[source::udp:514]
TRANSFORMS-bad_log = del_chHosts

transforms.conf

[del_chHosts]
REGEX = ^Host=rccr*
DEST_KEY = queue
FORMAT = nullQueue

We are looking to block off logs from host names starting with rccr. We are wondering if any one could point out what we could be doing wrong here?

We also tried a single host instead of wildcard, but still didn't work, so not sure. We'd appreciate your help.

Note: Logs are first forwarded to syslog server (splunk and syslog server is the same server) and then to Splunk.

Thank you

0 Karma

woodcock
Esteemed Legend

Like this:

props.conf:

[source::udp:514]
TRANSFORMS-bad_log = del_chHosts

transforms.conf

[del_chHosts]
SOURCE_KEY = MetaData:Host
REGEX = ^rccr*
DEST_KEY = queue
FORMAT = nullQueue

woodcock
Esteemed Legend

Did any solution work?

0 Karma

maciep
Champion

I'm still confused here.

It looks like that log sample you provided is from a Splunk internal log on the indexer maybe? It's just giving you metrics on what is being indexed. I don't think you want to use that as your base for filtering.

Do you have a sample of the actual log data you want to filter out. To apply the transforms you want, the regex has to match that data, not the log format of splunk's internal log.

For example, if you search for the following, what do those raw events look like?

index="main" sourcetype="syslog" host="rcmk"
0 Karma

AKG
Path Finder

After Looking through the available logs we tried with following changes, still no luck, are we missing something?

Props.conf

[source::tcp:11514]
TRANSFORMS-drop = del_cephHosts

OR

[source::syslog]
TRANSFORMS-drop = del_Hosts

transforms.conf

[del_Hosts]
REGEX = ^h="hostName"
DEST_KEY = queue
FORMAT = nullQueue

OR

[del_Hosts]
REGEX = h="hostname"
DEST_KEY = queue
FORMAT = nullQueue

OR

[del_Hosts]
REGEX = h="host*"
DEST_KEY = queue
FORMAT = nullQueue

OR

[del_Hosts]
REGEX = (?msi)^h="hostname"
DEST_KEY = queue
FORMAT = nullQueue
0 Karma

somesoni2
Revered Legend

Can you post some sample log entries? Also snipped from the inputs.conf ?

0 Karma

AKG
Path Finder

Here is the log i have access in splunk as license has been exceeded more than 5 times already.

09-19-2015 11:07:43.945 +1000 INFO LicenseUsage - type=Usage s="tcp:11514" st=syslog h="rcmk" o="" idx="main" i="3703752D-8AE6-4156-99CE-9B4A292A4D73" pool="auto_generated_pool_enterprise" b=4717246 poolsz=21474836480

also we are looking at doing this ate splunk server end as host is basically forwarding to syslog log server which is plunk server itself

------Input file-----
[default]
host = myhost.com

-----------end of input file-------------------

0 Karma

maciep
Champion

do the log entries actually have key-value pair in them like your regex does? Meaning, do the logs literally read "Host=rccrxxxx"? And if so, is that literally the first thing in the log (as opposed to a timestamp for example)?

Have you tried just putting rccr as the REGEX value?

0 Karma

AKG
Path Finder

Hi Maciep

If I just have rccr* as regex value than I am worried if it is going to block any log that has rccr.

We want to block out group of host only.

Thank you

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...