topic Re: How to filter syslog event level? in Getting Data In

How to filter syslog event level?

matoulas — Fri, 08 Nov 2019 18:37:33 GMT

Hi,

I've configured WTI device syslog to send to Splunk.
I can see all the syslog activity is there in Splunk Event.

My question is how do I filter syslog event level in Splunk- like syslog severity from WTI Event?

Example:
0 - Emergency, 1 - Alert, 2 - Critical, 3 - Error, 4 - Warning, 5 - notification, 6 - information and 7 - debugging.

Looking forward to your help.

Re: How to filter syslog event level?

gcusello — Sat, 09 Nov 2019 11:19:54 GMT

Hi matoulas,
as you can read at https://docs.splunk.com/Documentation/Splunk/8.0.0/Forwarding/Routeandfilterdatad , to filter data you have to find the regex to identify logs to discard.
In other words, if in your logs with sourcetype=my_sourcetype you have something like this

2019-11-09 12:05:59 10.10.10.10 my_host 2 message yf uif ouyf ouyf ouyf opiyf uo pgu pyi  yf yif i piyf puig piuf piuf pu pif piuf piyf

where the number after the hostname is the event level to use for filtering and you want to delete all the events with level=6 or 7, you have to find a regex like this:

^\d+-\d+-\d+\s+\d+:\d+:\d+\s\d+\.\d+\.\d+\.\d+\s\w+\s([6-7])

that you can test at https://regex101.com/r/UZRO64/1

Then you have to put on your indexers in props.conf file the following stanza:

[my_sourcetype]
TRANSFORMS-null = setnull

and in transforms.conf file the following stanza:

[setnull]
REGEX = ^\d+-\d+-\d+\s+\d+:\d+:\d+\s\d+\.\d+\.\d+\.\d+\s\w+\s([6-7])
DEST_KEY = queue
FORMAT = nullQueue

Then you have to restart Splunk on Indexers.
In this way the logs that match the above regex will be deleted.

If you use an Heavy Forwarder to ingest syslogs, you have to put the above props.conf and transforms.conf on the Heavy Forwarder instead that on Indexer.

If you can share a sample of your logs, I could help you better to find the correct regex.

Ciao.
Giuseppe

Re: How to filter syslog event level?

matoulas — Tue, 12 Nov 2019 21:13:38 GMT

See below for the WTI Syslog that send to Splunk.
I would like to see if I can filter by Syslog Severity level. I've no idea how to search for syslog Severity level in Splunk.

Nov 12 12:02:17 192.168.100.220 1 2019-11-12T12:00:24-08:00 CPM-1600-1-ECM server - - [meta sequenceId="196" enterpriseId="2634.1.17.16" vendorId="WTI"] CPM: CPM-1600-1-ECM, (AUDIT LOG) DATE-TIME: 11/12/19 12:00:24, USERNAME: SR-Turn-OFF-Odd Turn OFF plug B7

Nov 12 08:02:21 192.168.100.220 1 2019-11-12T08:00:28-08:00 CPM-1600-1-ECM server - - [meta sequenceId="186" enterpriseId="2634.1.17.16" vendorId="WTI"] CPM: CPM-1600-1-ECM, (AUDIT LOG) DATE-TIME: 11/12/19 08:00:28, USERNAME: SR-Reboot-Odd-Plug BOOT plug B3

Re: How to filter syslog event level?

gcusello — Wed, 13 Nov 2019 08:11:19 GMT

Hi @matoulas,
I don't know your logs, so what's the Syslog Severity level in your logs?
if it's the first number after date and IP address, you can use this regex:

^\w+\s+\d+\s+\d+:\d+:\d+\s\d+\.\d+\.\d+\.\d+\s([6-7])

that you can test at https://regex101.com/r/UZRO64/2
Ciao.
Giuseppe

Re: How to filter syslog event level?

matoulas — Wed, 30 Sep 2020 02:57:49 GMT

Updated I got it to work by set no_priority_stripping=true in input.conf in Splunk server and after install syslog priority filed decoder / lookup. I now be able to filter my severity level as I want.
https://splunkbase.splunk.com/app/736/

Thanks,
Matoula Senethavong

Re: How to filter syslog event level?

woodcock — Sun, 17 Nov 2019 01:14:36 GMT

You should not be sending syslog directly to Splunk. You should setup a syslog-ng server for this purpose:
http://www.georgestarcher.com/splunk-success-with-syslog/
As far as how to set it up, probably the best (and very new option) is here:
https://www.splunk.com/en_us/blog/tips-and-tricks/splunk-connect-for-syslog-turnkey-and-scalable-syslog-gdi.html
Now, once all that is done, you should do ALL of your filtering in syslog-ng, not Splunk.

Re: How to filter syslog event level?

matoulas — Wed, 11 Dec 2019 17:18:55 GMT

Thanks got it to work now.