topic How to search between two lines? in Splunk Search

How to search between two lines?

runiyal — Thu, 12 Jan 2023 20:39:01 GMT

Hello All,

I have following lines in the log file -

Server8 runiyal 2023-01-12 09:48:41,880 INFO Plugin.DOCUMENT Bytes size from input stream : 2072823 server8 runiyal 2023-01-12 09:48:41,978 INFO Plugin.DOCUMENT File size after upload to temp folder: 2072823 server8 runiyal 2023-01-12 09:48:43,391 SUCCESS Plugin.DOCUMENT File size after notifying the docrepo : 2072823

I want to -
1. Search for the DocID in the end <2072823>; It should have SUCCESS written in line. (Line3)
2. It should then look at the above line with string "from input stream" for the same DocID (Line 1)
3. Reduce the timestamp from SUCCESS line (3) to the timestamp in line with the text "from input stream" (Line 1) - Result will be in seconds
4. Result should be in two columns: "DocID" and "Time Taken" (4)

Will appreciate your inputs on how this can be achieved. Thanks!

Re: Searching between two lines

gcusello — Thu, 12 Jan 2023 16:45:45 GMT

Hi @runiyal,

I suppose that you already extracted the DocID field, otherwise you have to add a rex command before the stats command.

you could use the transaction command

index=your_index 1. (SUCCESS OR "from input stream" OR "from input stream") | transaction DocID | rename duration AS "Time Taken" | table DocID "Time Taken"

that's very slow, so try this different approach:

index=your_index 1. (SUCCESS OR "from input stream" OR "from input stream") | stats earliest(_time) AS earliest latest(_time) AS latest BY DocID | eval Time_Take=latest-earliest | table DocID Time_Taken | rename Time_Taken AS "Time Taken"

Ciao.

Giuseppe

Re: Searching between two lines

runiyal — Thu, 12 Jan 2023 18:51:38 GMT

Thanks Giuseppe.. I have to extract the docID field too.

Re: Searching between two lines

gcusello — Fri, 13 Jan 2023 07:09:24 GMT

Hi @runiyal,

in this case. you have to add a regex extraction to your search:

with transaction:

index=your_index 1. (SUCCESS OR "from input stream" OR "from input stream") | rex "(?<DocID>\d+)$" | transaction DocID | rename duration AS "Time Taken" | table DocID "Time Taken"

or with stats (better):

index=your_index 1. (SUCCESS OR "from input stream" OR "from input stream") | rex "(?<DocID>\d+)$" | stats earliest(_time) AS earliest latest(_time) AS latest BY DocID | eval Time_Take=latest-earliest | table DocID Time_Taken | rename Time_Taken AS "Time Taken"

You can test the regex at https://regex101.com/r/TgQtHA/1

Ciao.

Giuseppe