Getting Data In

How to filter syslog event level?

matoulas
Path Finder

Hi,

I've configured WTI device syslog to send to Splunk.
I can see all the syslog activity is there in Splunk Event.

My question is how do I filter syslog event level in Splunk- like syslog severity from WTI Event?

Example:
0 - Emergency, 1 - Alert, 2 - Critical, 3 - Error, 4 - Warning, 5 - notification, 6 - information and 7 - debugging.

Looking forward to your help.

matoulas
Path Finder

Thanks got it to work now.

0 Karma

woodcock
Esteemed Legend

You should not be sending syslog directly to Splunk. You should setup a syslog-ng server for this purpose:
http://www.georgestarcher.com/splunk-success-with-syslog/
As far as how to set it up, probably the best (and very new option) is here:
https://www.splunk.com/en_us/blog/tips-and-tricks/splunk-connect-for-syslog-turnkey-and-scalable-sys...
Now, once all that is done, you should do ALL of your filtering in syslog-ng, not Splunk.

0 Karma

matoulas
Path Finder

Updated I got it to work by set no_priority_stripping=true in input.conf in Splunk server and after install syslog priority filed decoder / lookup. I now be able to filter my severity level as I want.
https://splunkbase.splunk.com/app/736/

Thanks,
Matoula Senethavong

0 Karma

matoulas
Path Finder

See below for the WTI Syslog that send to Splunk.
I would like to see if I can filter by Syslog Severity level. I've no idea how to search for syslog Severity level in Splunk.

Nov 12 12:02:17 192.168.100.220 1 2019-11-12T12:00:24-08:00 CPM-1600-1-ECM server - - [meta sequenceId="196" enterpriseId="2634.1.17.16" vendorId="WTI"] CPM: CPM-1600-1-ECM, (AUDIT LOG) DATE-TIME: 11/12/19 12:00:24, USERNAME: SR-Turn-OFF-Odd Turn OFF plug B7

Nov 12 08:02:21 192.168.100.220 1 2019-11-12T08:00:28-08:00 CPM-1600-1-ECM server - - [meta sequenceId="186" enterpriseId="2634.1.17.16" vendorId="WTI"] CPM: CPM-1600-1-ECM, (AUDIT LOG) DATE-TIME: 11/12/19 08:00:28, USERNAME: SR-Reboot-Odd-Plug BOOT plug B3

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi matoulas,
as you can read at https://docs.splunk.com/Documentation/Splunk/8.0.0/Forwarding/Routeandfilterdatad , to filter data you have to find the regex to identify logs to discard.
In other words, if in your logs with sourcetype=my_sourcetype you have something like this

2019-11-09 12:05:59 10.10.10.10 my_host 2 message yf uif ouyf ouyf ouyf opiyf uo pgu pyi  yf yif i piyf puig piuf piuf pu pif piuf piyf 

where the number after the hostname is the event level to use for filtering and you want to delete all the events with level=6 or 7, you have to find a regex like this:

^\d+-\d+-\d+\s+\d+:\d+:\d+\s\d+\.\d+\.\d+\.\d+\s\w+\s([6-7])

that you can test at https://regex101.com/r/UZRO64/1

Then you have to put on your indexers in props.conf file the following stanza:

[my_sourcetype]
TRANSFORMS-null = setnull

and in transforms.conf file the following stanza:

[setnull]
REGEX = ^\d+-\d+-\d+\s+\d+:\d+:\d+\s\d+\.\d+\.\d+\.\d+\s\w+\s([6-7])
DEST_KEY = queue
FORMAT = nullQueue

Then you have to restart Splunk on Indexers.
In this way the logs that match the above regex will be deleted.

If you use an Heavy Forwarder to ingest syslogs, you have to put the above props.conf and transforms.conf on the Heavy Forwarder instead that on Indexer.

If you can share a sample of your logs, I could help you better to find the correct regex.

Ciao.
Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @matoulas,
I don't know your logs, so what's the Syslog Severity level in your logs?
if it's the first number after date and IP address, you can use this regex:

^\w+\s+\d+\s+\d+:\d+:\d+\s\d+\.\d+\.\d+\.\d+\s([6-7])

that you can test at https://regex101.com/r/UZRO64/2
Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...