Splunk Search
Highlighted

Can we include OR/AND operator in a transaction

Explorer

I have the following log sets, one for success case and one for the failure case

Success:
id=11111 msg=Begin process...
id=11111 msg=check
id=11111 msg=Success...

failure:
id=22222 msg=Begin process...
id=22222 msg=check
id=22222 msg=Fail...

Here I want to check the time between the events using the transaction.

host=* sourcetype=** source="*/example.log" "Begin process*" OR "Success*"
  | transaction traceId startswith="Begin process" endswith="Success" 
  | table traceId duration _time

Above query will give me the transactions of a success case only.
Can we use AND Operator in the endswith so that I can check the duration between events irrespective of it being a success or failure?

0 Karma
Highlighted

Re: Can we include OR/AND operator in a transaction

Legend

@amunag439 I would recommend using stats instead of transaction for this scenario

 <yourCurrentCodeToGetTimeMsgAndID>
|  stats list(msg) as msg_all values(msg) as msg_unique min(_time) as _time max(_time) as latest_time by id
|  eval duration=latest_time-_time
|  fields - latest_time
|  eval startswith=mvindex(msg_all,0),endswith=mvindex(msg_all,mvcount(msg_all)-1)
|  fields - msg_all msg_unique
|  search startswith="Begin Process" endswith="*"

Following is a run anywhere example based on sample data and details provided. It looks only for startswith condition. Endwith any value is accepted (you can explicitly set to Success and Fail as well if there are only two final messages in your data).

|  makeresults
|  eval _time=relative_time(now(),"-1d")
|  eval data="id=11111 msg=Begin process;id=11111 msg=check;id=11111 msg=Success;id=22222 msg=Begin process;id=22222 msg=check;id=22222 msg=Fail"
|  makemv data delim=";"
|  mvexpand data
|  rename data as _raw
|  KV
|  eval msg=replace(msg,"Begin","Begin Process")
|  eval delta_duration=random()
|  eval delta_duration=substr(delta_duration,1,3)
|  accum delta_duration
|  eval _time=_time+delta_duration
|  fields - _raw delta_duration
|  stats list(msg) as msg_all values(msg) as msg_unique min(_time) as _time max(_time) as latest_time by id
|  eval duration=latest_time-_time
|  fields - latest_time
|  eval startswith=mvindex(msg_all,0),endswith=mvindex(msg_all,mvcount(msg_all)-1)
|  fields - msg_all msg_unique
|  search startswith="Begin Process" endswith="*"

View solution in original post

Highlighted

Re: Can we include OR/AND operator in a transaction

Explorer

@niketnilay in the failure case what if there are few more logs? if Fail is not a final log, how do we approach that?

0 Karma
Highlighted

Re: Can we include OR/AND operator in a transaction

Legend

@amunag439 the query that I have provided is similar to what transaction will do when there is starts with condition but ends with can be anything.

If you want other more specific conditions you will have to play with msg_unique which has all distinct values of msg field present for specific ids. Final 2 pipes need to change as follows.

  |  fields - msg_all
  |  search startswith="Begin Process" endswith="*"

msgunique=Success AND msgunique=Fail.

0 Karma
Highlighted

Re: Can we include OR/AND operator in a transaction

Esteemed Legend

There is only one use case where the use of transaction is merited but this command scales so poorly that I am not even going to mention it. Stop using transaction; try this:

host=* sourcetype=** source="*/example.log" "Begin process*" OR "Success*" OR "Failure*"
| streamstats count(eval(searchmatch("Success* OR Failure*"))) AS sessionID BY traceId
| stats range(_time) AS duration list(_raw) AS events min(_time) AS _time BY traceId sessionID
0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.