Splunk Search

Number field extraction from the log and need to find difference of the next number

nramya82
Explorer

Hi ,

I have below format logs which gets generated every 15 minutes in the below pattern and i need to find out the number request hitting the server for every 15 minutes.I was trying to extract number field and trying to get difference of the next number so that I can get generate a graph which will display number of request hitting server for every 15 minutes

0140928.log:2014/09/28 06:45:00,911 Info   CS Traffic: secureserver ABC:  2394528 / 000 

20140928.log:2014/09/28 07:00:00,910 Info   CS Traffic: secureserver ABC:  2397206 / 000 

My query is not working. I am not sure why its not working . Can some one please help?.

|"secureserver ABC:" | rex field=secureserver ABC: mode=sed "s/[^0-9]//g" | delta count(secureserver ABC) AS diff  
Tags (3)
0 Karma

MuS
SplunkTrust
SplunkTrust

hmm you're replacing instead of grabbing the numbers?? try this:

"secureserver ABC:" | rex "\:\s+(?<myNumber>\d+)\s+\/" | delta myNumber  

nramya82
Explorer

Thanks now i understood the field name purpose looks like we have to assign variable for the search field from where I have to extract the log and then do regex on the assigned variable but the issue is this number is a dynamic and it keeps getting updated for every 15 minutes my ultimate goal is to find the difference and graph that count in a nice graph.

2014/09/28 08:45:00,910 Info   CS Traffic: SecureServer ABC: 2412182 / 000
2014/09/28 09:00:00,912 Info   CS Traffic: SecureServer ABC: 2413791 / 000

The below query doesnt return the number so that i can perform difference of the next number

head 1 |eval foo="0140928.log:2014/09/28 06:45:00,911 Info CS Traffic: secureserver ABC: 2394528 / 000" | rex field=foo "\:\s+(?\d{7})\s+\/" | table myNum

Tried below option too

eval foo="*.log:* Info CS Traffic: secureserver ABC:"| rex field=foo "\:\s+(?\d{7})\s+\/" | table myNum
0 Karma

MuS
SplunkTrust
SplunkTrust

Look, I did provide you an example how it works - if you keep changing your searches to strings that cannot match, you will never get it working.

Read the docs on the search basics http://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/WelcometotheSearchTutorial and use this regex to get it working | rex ":\s\+(?\d+)\s+/"

I'm off this topic for now.......

nramya82
Explorer

I am not able to get just the number from the search On a unix command prompt with grep option,sed i amsimply getting just number from the log but the same thing I try to do search in the splunk its not working . I am totally stuck and not able to proceed further. Can some one please help?

In unix command prompt i can get the same number 2394528 by following command and same one is not working in splunk

sed s~^.: *([0-9]) /.*$~\1~

0140928.log:2014/09/28 06:45:00,911 Info CS Traffic: secureserver ABC: 2394528 / 000

secureserver ABC:" | rex ":\s+(?\d{7}\s+\/" -- doesnt give the number

0 Karma

MuS
SplunkTrust
SplunkTrust

This works like a charm:

index=_internal | head 1 |  eval foo="0140928.log:2014/09/28 06:45:00,911 Info CS Traffic: secureserver ABC: 2394528 / 000" | rex field=foo "\:\s+(?<myNum>\d{7})\s+\/" | table myNum
0 Karma

nramya82
Explorer

Thanks for the input For below format log

0140928.log:2014/09/28 06:45:00,911 Info CS Traffic: secureserver ABC: 2394528 / 000
20140928.log:2014/09/28 07:00:00,910 Info CS Traffic: secureserver ABC: 2397206 / 000 

I tried "secureserver ABC:" | rex "\:\s+(?\d{7}\s+\/)" just to see if i get the number and its not working. actually I tried below
"secure ABC: | rex "\s+\K\d{7}(?=\s+\/)" and it gives me missing group name not sure what i am doing here.
I think in this rex "\:\s+(?\d{7}\s+\/)" groupname is MyNumber and can be any name right ?.

0 Karma

MuS
SplunkTrust
SplunkTrust

No, you did not provide any group name in your regex's. Just try this as a start:

your base search here that provides the needed events | rex field=_raw ":\s\+(?<myNumber>\d+)\s+/" | delta myNumber 

PS: Don't search for your base search here that provides the needed events this is just a place holder 😉

0 Karma

nramya82
Explorer

Well I am getting " Error in 'SearchParser': Missing a search command before '"' and doesnt return any value. Can some one please help,??
Below is my search query

"SecureServer PPS" | "secureserver ABC:" | rex ":\s+(?\d+)\s+\/" | delta myNumber

0 Karma

MuS
SplunkTrust
SplunkTrust

I think you should start here http://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/WelcometotheSearchTutorial ....

but as a hint try this :

"SecureServer PPS" OR "secureserver ABC:" | rex ":\s\+(?<myNumber>\d+)\s+/" | delta myNumber 
0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...