topic Re: How to count and filter types of error data that are in the form of strings, not fields? in Splunk Search

How to count and filter types of error data that are in the form of strings, not fields?

shingdayho — Mon, 03 Nov 2014 18:41:52 GMT

Hi,

So I'm running a command which displays me errors (Aborted, Ping too slow etc, connection aborted), these are just strings of data, not fields.

I want to count how many of each error I get on a 7 day period. I am able to count how many in total there are, however as the data I need to filter is just a string of data, not a field I'm having some difficulties.

Thanks,

Re: How to count and filter types of error data that are in the form of strings, not fields?

somesoni2 — Mon, 03 Nov 2014 19:00:06 GMT

You would have to extract a field containing the error message and then you can count individual error message count. Please post some sample log entries, possibly covering all possible error messages and Splunkers here can help you find regex to extract the field.

Re: How to count and filter types of error data that are in the form of strings, not fields?

gkanapathy — Mon, 03 Nov 2014 19:04:40 GMT

You would either need to define a field to differentiate the types. Or you can use the "Patterns" tab to have Splunk generate event types to help differentiate different patterns of event text.

Re: How to count and filter types of error data that are in the form of strings, not fields?

shingdayho — Mon, 03 Nov 2014 19:06:24 GMT

Search thus far which shows all errors:
index=nagios AND "backuppc" AND "WARNING;HARD" AND "CURRENT SERVICE"

Some example results:

03/11/2014 00:00:00.000 [1414972800] CURRENT SERVICE STATE: backuppc.shingdayho;BACKUPPC;WARNING;HARD;3;BACKUPPC WARNING - (webserver.shingdayho (ping too slow: 182.5msec (threshold is 60msec)), teamspeak.shingdayho (ping too slow: 145.4msec (threshold is 60msec)), ) host = nagios.shingdayho.com index = nagios source = /var/log/nagios/nagios.log 
03/11/2014 00:00:00.000 [1414972800] CURRENT SERVICE STATE: backuppc.shingdayho;BACKUPPC;WARNING;HARD;3;BACKUPPC WARNING - (mail2.shingdayho (aborted by signal=PIPE), ) host = nagios.shingdayho.com index = nagios source = /var/log/nagios/nagios.log 
02/11/2014 00:00:00.000 [1414886400] CURRENT SERVICE STATE: backuppc.shingdayho;BACKUPPC;WARNING;HARD;3;BACKUPPC WARNING - (webserver.shingdayho (lost network connection during backup), ) host = nagios.shingdayho.com index = nagios source = /var/log/nagios/nagios.log 
01/11/2014 00:00:00.000 [1414800000] CURRENT SERVICE STATE: backuppc.shingdayho;BACKUPPC;WARNING;HARD;3;BACKUPPC WARNING - (mail1.shingdayho (aborted by signal=PIPE), )

Re: How to count and filter types of error data that are in the form of strings, not fields?

shingdayho — Mon, 03 Nov 2014 19:07:24 GMT

Could you please provide me to some examples which I could take a look at and I'll see if I can manipulate them for my needs, as well there is no "Patterns" tab in my Splunk, is there any other way to make Splunk generate these event types?

Re: How to count and filter types of error data that are in the form of strings, not fields?

somesoni2 — Mon, 03 Nov 2014 19:21:06 GMT

Try something like this

index=nagios AND "backuppc" AND "WARNING;HARD" AND "CURRENT SERVICE"  | rex "WARNING [^\(]*\([^\(]*\((?<ErrorMessage>[^=\:\),]*)" | stats count by ErrorMessage

Re: How to count and filter types of error data that are in the form of strings, not fields?

shingdayho — Mon, 03 Nov 2014 19:23:05 GMT

That works! Thank you for your quick replies and for helping me fix it!