topic Re: Need to print response time of request in Dashboards & Visualizations

Need to print response time of request

ajaynaralikar — Tue, 29 Sep 2020 20:35:55 GMT

Hi ,

I want create one table of three columns like below
Request_time Response_time Difference(Response_time - Request_time )
And my application logs are printed in below manner.

24-07-2018 09:41:47:111 [RetrieveInterestRate][INFO ]: |BEM_Gateway_Request_MF Request Processing Summary(TransactionId-**************) SCVT Recieved Request at 24-07-2018 09:41:47:108
24-07-2018 09:41:47:239 [RetrieveInterestRate][INFO ]: |BEM_Gateway_Response_MF Request Processing Summary(TransactionId-**************) SCVT sent response back to consumer at 24-07-2018 09:41:47:236

Here request time printed after SCVT Recieved Request at and response time printed after SCVT sent response back to consumer at
SO can please help here ?

Re: Need to print response time of request

skoelpin — Tue, 24 Jul 2018 14:10:28 GMT

Do you have a unique identifier that ties the request and response together? If so then you can tie these together and use a simple eval to do the math

Re: Need to print response time of request

adonio — Tue, 24 Jul 2018 14:30:35 GMT

hello there,

as @skoelpin suggested, a unique identifier will help you to tie the events together.
i mimicked an identifier for the following example, after the Summary(TransactionId-) in this case Summary(TransactionId-123)

try the following search anywhere:

| makeresults count=1 
| eval data = "24-07-2018 09:41:47:111 [RetrieveInterestRate][INFO ]: |BEM_Gateway_Request_MF Request Processing Summary(TransactionId-123) SCVT Recieved Request at 24-07-2018 09:41:47:108
; 24-07-2018 09:41:47:239 [RetrieveInterestRate][INFO ]: |BEM_Gateway_Response_MF Request Processing Summary(TransactionId-123) SCVT sent response back to consumer at 24-07-2018 09:41:47:236"
| makemv delim=";" data 
| mvexpand data
| rex field=data "SCVT\s+(?<request_or_response>[^\s]+).+at\s+(?<time>\d{2}\-\d{2}\-\d{4}\s+\d{2}\:\d{2}\:\d{2}\:\d{3})"
| rex field=data "TransactionId\-(?<transaction_id>\S+)\)"
| eval time_epoch = strptime(time, "%d-%m-%Y %H:%M:%S:%3N")
| eval recieved_time = if(request_or_response="Recieved",time_epoch,null())
| eval response_time = if(request_or_response="sent",time_epoch,null())
| stats values(recieved_time) as rec_time values(response_time) as res_time by transaction_id
| eval duration_in_seconds = res_time - rec_time

hope it helps

Re: Need to print response time of request

ajaynaralikar — Tue, 29 Sep 2020 20:38:37 GMT

Thanks @adonio n @skoelpin

I tried the command which is given by adonio and it works successfully for manual entry(makeresults value). But whenever I am dealing with actual file it showing empty data. PFB my command,

bemLog=RetrieveInterestRate | eval Event = "RetrieveInterestRate" | makemv delim=";" Event | mvexpand Event | rex field=Event "SCVT\s+(?[^\s]+).+at\s+(?\d{2}-\d{2}-\d{4}\s+\d{2}:\d{2}:\d{2}:\d{3})" | rex field=Event "TransactionId-(?\S+))" | eval time_epoch = strptime(time, "%d-%m-%Y %H:%M:%S:%3N") | eval recieved_time = if(request_or_response="Recieved",time_epoch,null()) | eval response_time = if(request_or_response="sent",time_epoch,null()) | stats values(recieved_time) as rec_time values(response_time) as res_time by transaction_id | eval duration_in_seconds = res_time - rec_time

bemLog=RetrieveInterestRate printing the logs in below format
Time Event
7/25/18 25-07-2018 10:47:46:680 [RetrieveInterestRate][INFO ]: |BEM_Gateway_Response_MF Request Processing Summary(Transaction Id-ABC123)
10:47:46.680 AM SCVT sent response back to consumer at 25-07-2018 10:47:46:671

7/25/18 5-07-2018 10:47:46:540 [RetrieveInterestRate][INFO ]: |BEM_Gateway_Request_MF Request Processing Summary(TransactionId-ABC123)
10:47:46.540 AM SCVT Recieved Request at 25-07-2018 10:47:46:537

Re: Need to print response time of request

ajaynaralikar — Wed, 25 Jul 2018 12:12:50 GMT

I got issue and issue with command and it is related to delimiter and in my logs delimiter is new line.
So can you tell me how to set parameter to new line like command makemv delim=";"
Also I searched and get to know that there is know mechanism to set parameter for \n.

Re: Need to print response time of request

skoelpin — Wed, 25 Jul 2018 13:49:36 GMT

As discussed earlier, you need a unique identifier for the request response pair so you can tie them together and do the math on the timestamp.. @Adonio example is a run-anywhere example which uses | makeresults to add that unique identifier. Until you have the unique identifier for each pair, you cannot do it

Re: Need to print response time of request

adonio — Wed, 25 Jul 2018 13:57:56 GMT

the first 5 lines in my command only generating fake data
try to replace them with a search that finds events: index = <some_index> sourcetype = <some_sourcetype> ...
also remove the field=data from the rex command
pay attention to your data, do you have a unique identifier that will help tie down events together?
if not how can you tell which "response" goes with which "received"