How to filter out syslog events from a group of hosts via regex?

[AKG]

Hi

We have a group of servers and looks like they have been reconfigured. Until we get hold of a sysadmin and fix the issue, we have to drop off logs from those servers.

Below is how we tried filtering with no success.

props.conf

[source::udp:514]
TRANSFORMS-bad_log = del_chHosts

transforms.conf

[del_chHosts]
REGEX = ^Host=rccr*
DEST_KEY = queue
FORMAT = nullQueue

We are looking to block off logs from host names starting with rccr. We are wondering if any one could point out what we could be doing wrong here?

We also tried a single host instead of wildcard, but still didn't work, so not sure. We'd appreciate your help.

Note: Logs are first forwarded to syslog server (splunk and syslog server is the same server) and then to Splunk.

Thank you

[woodcock]

Like this:

props.conf:

[source::udp:514]
TRANSFORMS-bad_log = del_chHosts

transforms.conf

[del_chHosts]
SOURCE_KEY = MetaData:Host
REGEX = ^rccr*
DEST_KEY = queue
FORMAT = nullQueue

[woodcock]

Did any solution work?

[maciep]

I'm still confused here.

It looks like that log sample you provided is from a Splunk internal log on the indexer maybe? It's just giving you metrics on what is being indexed. I don't think you want to use that as your base for filtering.

Do you have a sample of the actual log data you want to filter out. To apply the transforms you want, the regex has to match that data, not the log format of splunk's internal log.

For example, if you search for the following, what do those raw events look like?

index="main" sourcetype="syslog" host="rcmk"

[AKG]

After Looking through the available logs we tried with following changes, still no luck, are we missing something?

Props.conf

[source::tcp:11514]
TRANSFORMS-drop = del_cephHosts

OR

[source::syslog]
TRANSFORMS-drop = del_Hosts

transforms.conf

[del_Hosts]
REGEX = ^h="hostName"
DEST_KEY = queue
FORMAT = nullQueue

OR

[del_Hosts]
REGEX = h="hostname"
DEST_KEY = queue
FORMAT = nullQueue

OR

[del_Hosts]
REGEX = h="host*"
DEST_KEY = queue
FORMAT = nullQueue

OR

[del_Hosts]
REGEX = (?msi)^h="hostname"
DEST_KEY = queue
FORMAT = nullQueue

[somesoni2]

Can you post some sample log entries? Also snipped from the inputs.conf ?

[AKG]

Here is the log i have access in splunk as license has been exceeded more than 5 times already.

09-19-2015 11:07:43.945 +1000 INFO LicenseUsage - type=Usage s="tcp:11514" st=syslog h="rcmk" o="" idx="main" i="3703752D-8AE6-4156-99CE-9B4A292A4D73" pool="auto_generated_pool_enterprise" b=4717246 poolsz=21474836480

also we are looking at doing this ate splunk server end as host is basically forwarding to syslog log server which is plunk server itself

------Input file-----
[default]
host = myhost.com

-----------end of input file-------------------

[maciep]

do the log entries actually have key-value pair in them like your regex does? Meaning, do the logs literally read "Host=rccrxxxx"? And if so, is that literally the first thing in the log (as opposed to a timestamp for example)?

Have you tried just putting rccr as the REGEX value?

[AKG]

Hi Maciep

If I just have rccr* as regex value than I am worried if it is going to block any log that has rccr.

We want to block out group of host only.

Thank you