How to calculate duration for repeated event by unique string

[nraf]

Hi,

I am new to Splunk, just started for few days.  Below is the events that I have searched and sorted, I would like to get the duration between step 1 and step 2 by the same UniqueString, and show it in a new field. The reason to store it in a new field is because I would like to later make a chart to show the before/after 2 eval run. I read many helps and I tried eval, stats but it ends up 0 result. Please help.

My search: index=aaa host=aaa*  sourcetype=aaa_logs Command="Step1*" OR Command="Step2*" | sort by _time
| dedup UniqueString

210312 12:07:45.619 INFO Step1( "UniqueString2.DAT" )
210312 12:07:55.609 INFO Step1( "UniqueString1.DAT" )
210312 12:07:56.015 INFO Step2("M;UniqueString1", "A", "C", "D", "A")
210312 12:07:56.609 INFO Step1( "UniqueString3.DAT" )
210312 12:15:27.989 INFO Step2("M;UniqueString2", "B", "E", "F", "B")

[nraf]

I used below search but it doesn't pair the Step1 and Step2 UniqueString.

index=aaa host=aaa* sourcetype=aaa_logs Command="Step1*" OR Command="Step2*" | sort by _time
| dedup UniqueString
|sort Date,Time, UniqueString
|transaction startswith=Step1* endswith=Step2* |table UniqueString duration

The result is not correct:

For Step1 and Step2 that having same Date Time, they are paired correctly by the UniqueString:

210312 14:07:55.609 INFO Step1( "UniqueString7.DAT" )
210312 14:07:55.609 INFO Step2("M;UniqueString7", "A", "C", "D", "A")

However, for Step1 and Step2 that has different Date Time, they look like this:

210312 12:07:56.015 INFO Step2("M;UniqueString6", "A", "C", "D", "A")
210312 12:06:56.609 INFO Step1( "UniqueString5.DAT" )

210312 12:05:56.015 INFO Step2("M;UniqueString5", "A", "C", "D", "A")
210312 12:04:56.609 INFO Step1( "UniqueString4.DAT" )

I think the problem is due to not every Step1 will have the corresponding Step2. Please help!

Resolved the issue by removing the | sort by _time

[nraf]

index=aaa host=aaa* sourcetype=aaa_logs Command="Step1*" OR Command="Step2*" | dedup UniqueString
|transaction startswith=Step1* endswith=Step2* |table date_mday UniqueString duration
| sort _time

Above is my search string, if I search the last 7 days, the events are the same as the last 24 hours search result, which is impossible. Last 7 days should have more records.

Please help~

[bowesmana]

Using the transaction command can have unexpected behaviour, as it is constrained by memory and has limitations.

The documentation for transaction suggests you use stats as an alternative

https://docs.splunk.com/Documentation/Splunk/8.1.2/Search/Abouttransactions

| stats range(_time) as duration by UniqueString

This would give you the duration.

Not sure what your dedup command is for as that would prevent the above from working and it's not clear what fields you have,  but this should work.

Here's a run anywhere example - all bar the last line is setting up your example data

| makeresults
| eval event="210312 12:07:45.619 INFO Step1( \"UniqueString2.DAT\" )###210312 12:07:55.609 INFO Step1( \"UniqueString1.DAT\" )###210312 12:07:56.015 INFO Step2(\"M;UniqueString1\", \"A\", \"C\", \"D\", \"A\")###210312 12:07:56.609 INFO Step1( \"UniqueString3.DAT\" )###210312 12:15:27.989 INFO Step2(\"M;UniqueString2\", \"B\", \"E\", \"F\", \"B\")"
| makemv event delim="###"
| mvexpand event
| rex field=event "(?<t>\d+ \d+:\d+:\d+\.\d+) INFO (?<Command>\w+).*(?<UniqueString>UniqueString\d+)"
| eval _time=strptime(t, "%y%m%d %H:%M:%S.%Q")
| table _time Command UniqueString
| stats range(_time) as duration list(_time) as times by UniqueString