Splunk Search

How to count the difference in a rolling number in events across multiple hosts?

Rob
Splunk Employee
Splunk Employee

The following example events are indexed by Splunk:

Dec 1 00:47:58 serverName data-collector[1234]: #A_RECV# 1234, 5
Dec 1 00:47:58 serverName data-collector[1234]: #B_RECV# 5678, 10
Dec 1 00:47:58 serverName data-collector[1234]: #C_RECV# 2587, 100
Dec 1 00:47:58 serverName data-collector[1234]: #D_RECV# 9852, 150

What I would like to do is for a search that can show a table including the host name (A_RECV, B_RECV, etc.) the number of requests (first number) and the number of errors (second number).

Both the number of requests and the number of errors are incremental so therefore the search will need to show the difference between the number of requests from the previous event to the last event if there has been a change. An example of the events from host A might be:

Dec 1 00:47:58 serverName data-collector[1234]: #A_RECV# 1234, 5
Dec 1 00:53:58 serverName data-collector[1234]: #A_RECV# 1240, 7

If so, then the table should show that Host A had 6 requests and 2 errors.

Tags (2)
1 Solution

Rob
Splunk Employee
Splunk Employee

I believe you may want to use something similar to the following search.

host="hostName" | rex "(?P<IDLetter>[A-Z]).RECV..(?P<Requests>\\d+),.(?P<ErrorCount>\\d+)" | eval time=_time| transaction fields=IDLetter maxspan=60m | stats range(ErrorCount) range(Requests) range(time) by IDLetter

The search will first break up the string by using regex to extract the fields we want to pay attention to followed by getting the UTC time stamp and then piping that to only show the events for the last 60min. Then its just a final pipe to the stats in order to work the math to find the differences between the error count, the requests and the time.

View solution in original post

Rob
Splunk Employee
Splunk Employee

I believe you may want to use something similar to the following search.

host="hostName" | rex "(?P<IDLetter>[A-Z]).RECV..(?P<Requests>\\d+),.(?P<ErrorCount>\\d+)" | eval time=_time| transaction fields=IDLetter maxspan=60m | stats range(ErrorCount) range(Requests) range(time) by IDLetter

The search will first break up the string by using regex to extract the fields we want to pay attention to followed by getting the UTC time stamp and then piping that to only show the events for the last 60min. Then its just a final pipe to the stats in order to work the math to find the differences between the error count, the requests and the time.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...