Can anyone help me with a query that detects when a page takes longer than 30 seconds to load?

[satyaallaparthi]

Can anyone help me with a query that detects when a page takes longer than 30 seconds to load? I got URL extraction, and I want to know how to get startTime and endTime into results to get duration from that event, which is in JSON format.

{"Name":"url","Value":"\"/Services/****/*******-Services-DynamicMenuService/GetMenuForOutlet\""}{"Name":"duration","Value":"574"},{"Name":"logSource","Value":"\"HttpInterceptor\""},{"Name":"result","Value":"\"failed\""},{"Name":"startTime","Value":"\"2019-03-01T19:07:30.368Z\""},{"Name":"endTime","Value":"\"2019-03-01T19:07:30.942Z\""},{"Name":"createDate","Value":"\"2019-03-01T19:07:30.942Z\""},{"Name":"body","Value":"\"   

[somesoni2]

Give this a try

your base search
| rex "startTime[^\:]+\:[^\d]+(?<startTime>[^Z]+Z).+?endTime[^\:]+\:[^\d]+(?<endTime>[^Z]+Z)"
|eval startTime= strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 |eval endTime= strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 | eval duration = endTime-startTime
| table duration

[somesoni2]

How are you extracting field url? You should be able to use the same method to extract startTime and endTime fields (regex will be updated of course). Then you can use strptime function to convert string formatted date to epoch to calculate duration/response time.
https://answers.splunk.com/answers/454902/how-to-calculate-the-duration-of-a-single-event.html

[satyaallaparthi]

If am extracting duration from IFX.. its not extracting for all events properly.. we need startTime and endTime but I can't able to extract those.

[chrisyounger · Accepted solution]

Try this:

| your search
| rex "startTime\",\"Value\":\"\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\"(?<endTime>[\d\-T:\.Z]*)"
|eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
|eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
| eval duration = end - start
| search duration > 30

All the best

[satyaallaparthi]

Error in 'rex' command: Encountered the following error while compiling the regex 'startTime","Value":"\"(?[^]).?endTime","Value":"\"(?[^]*)': Regex: missing terminating ] for character class.

The search job has failed due to an error. You may be able view the job in the Job Inspector.

getting the above error and not allowing me to go further.

Thanks!

[chrisyounger]

I have updated my answer. Can you give it another try?

[satyaallaparthi]

Error in 'SearchParser': Missing a search command before '\'. Error at position '142' of search query 'search index=* sourcetype="mysrctpe"..{snipped} {errorcontext = tartTime>[\d-T:.Z]*}'.

again error

[chrisyounger]

OK this one should work:

your search
| rex "startTime\",\"Value\":\"\\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\\"(?<endTime>[\d\-T:\.Z]*)"
 |eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 |eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
 | eval duration = end - start
 | search duration > 30

[satyaallaparthi]

no result found .. if I am doing table duration.. that means not extracting properly right ?

I got zero result.. please help me with that..

Thanks,
Satya

[chrisyounger]

What about this:

 your search
 | rex "startTime\",\"Value\":\"\\\"(?<startTime>[\d\-T:\.Z]*).*?endTime\",\"Value\":\"\\\"(?<endTime>[\d\-T:\.Z]*)"
  |eval start = strptime(startTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
  |eval end = strptime(endTime, "%Y-%m-%dT%H:%M:%S.%3QZ")
  | eval duration = end - start
| table *