Getting Data In

Can anyone help me with a query that detects when a page takes longer than 30 seconds to load?

satyaallaparthi
Communicator

Can anyone help me with a query that detects when a page takes longer than 30 seconds to load? I got URL extraction, and I want to know how to get startTime and endTime into results to get duration from that event, which is in JSON format.

{"Name":"url","Value":"\"/Services/****/*******-Services-DynamicMenuService/GetMenuForOutlet\""}{"Name":"duration","Value":"574"},{"Name":"logSource","Value":"\"HttpInterceptor\""},{"Name":"result","Value":"\"failed\""},{"Name":"startTime","Value":"\"2019-03-01T19:07:30.368Z\""},{"Name":"endTime","Value":"\"2019-03-01T19:07:30.942Z\""},{"Name":"createDate","Value":"\"2019-03-01T19:07:30.942Z\""},{"Name":"body","Value":"\"   
0 Karma
1 Solution

chrisyounger
SplunkTrust
SplunkTrust

Try this:

| your search
| rex "startTime\",\"Value\":\"\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\"(?<endTime>[\d\-T:\.Z]*)"
|eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
|eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
| eval duration = end - start
| search duration > 30

All the best

View solution in original post

0 Karma

somesoni2
Revered Legend

Give this a try

your base search
| rex "startTime[^\:]+\:[^\d]+(?<startTime>[^Z]+Z).+?endTime[^\:]+\:[^\d]+(?<endTime>[^Z]+Z)"
|eval startTime= strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 |eval endTime= strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 | eval duration = endTime-startTime
| table duration
0 Karma

somesoni2
Revered Legend

How are you extracting field url? You should be able to use the same method to extract startTime and endTime fields (regex will be updated of course). Then you can use strptime function to convert string formatted date to epoch to calculate duration/response time.
https://answers.splunk.com/answers/454902/how-to-calculate-the-duration-of-a-single-event.html

0 Karma

satyaallaparthi
Communicator

If am extracting duration from IFX.. its not extracting for all events properly.. we need startTime and endTime but I can't able to extract those.

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

Try this:

| your search
| rex "startTime\",\"Value\":\"\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\"(?<endTime>[\d\-T:\.Z]*)"
|eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
|eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
| eval duration = end - start
| search duration > 30

All the best

0 Karma

satyaallaparthi
Communicator

Error in 'rex' command: Encountered the following error while compiling the regex 'startTime","Value":"\"(?[^]).?endTime","Value":"\"(?[^]*)': Regex: missing terminating ] for character class.

The search job has failed due to an error. You may be able view the job in the Job Inspector.

getting the above error and not allowing me to go further.

Thanks!

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

I have updated my answer. Can you give it another try?

0 Karma

satyaallaparthi
Communicator

Error in 'SearchParser': Missing a search command before '\'. Error at position '142' of search query 'search index=* sourcetype="mysrctpe"..{snipped} {errorcontext = tartTime>[\d-T:.Z]*}'.

again error

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

OK this one should work:

your search
| rex "startTime\",\"Value\":\"\\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\\"(?<endTime>[\d\-T:\.Z]*)"
 |eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 |eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 | eval duration = end - start
 | search duration > 30
0 Karma

satyaallaparthi
Communicator

no result found .. if I am doing table duration.. that means not extracting properly right ?

I got zero result.. please help me with that..

Thanks,
Satya

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

What about this:

 your search
 | rex "startTime\",\"Value\":\"\\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\\"(?<endTime>[\d\-T:\.Z]*)"
  |eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
  |eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
  | eval duration = end - start
| table *
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...