topic Guarantee startswith when using keepevicted in Splunk Search

Guarantee startswith when using keepevicted

gp2 — Tue, 04 Feb 2020 19:44:01 GMT

Hi Splunk community, I am trying to make a query that returns all transactions for a starting event and ending event that last a certain duration as well as any starting events that don't have an ending event for a specific time range. I attempted to do this by putting keepevicted = true in my transaction but this appears to include some unwanted data as well. I believe the below example will show what I mean:

The data list is as follows:
1. Connection: misc.
2. Connection: misc.
3. unneeded data
4. Connection: lost
5. Connection: finding
6. Connection: found
7.unneeded data
8. Connection: lost
9. Connection: finding
10. Connection: still finding

My query will be as follows
"Connection" | transaction startswith="lost " endswith="found" keepevicted=true

This will return 3 result transactions, events 1-2, events 4-6, and events 8-10. The last two are the ones I want but the first transaction is unneeded but shows up anyway as a result of keepevicted since they are considered close enough. If I removed keepevicted, I will only receive events 4-6 since 8-10 doesn't have the end event.

Is there a way I can modify the query so I receive the last two transactions but not the first one? Is it possible that transactions aren't necessary and there are other splunk commands that can get me what I want?

Re: Guarantee startswith when using keepevicted

to4kawa — Sat, 08 Feb 2020 02:02:47 GMT

|makeresults
| eval _raw="1. Connection: misc.
2. Connection: misc.
3. unneeded data
4. Connection: lost
5. Connection: finding
6. Connection: found
7.unneeded data
8. Connection: lost
9. Connection: finding
10. Connection: still finding"
| makemv delim="
" _raw
| stats count by _raw
| sort _raw
`comment("this is your sample, from here, the logic")`
| search "Connection"
| streamstats count(eval(searchmatch("lost"))) as session
| where session > 0

I don't use transaction. but it works.

Note that the older the log, the higher it must be.
If not, use | reverse before streamstats.

Re: Guarantee startswith when using keepevicted

gp2 — Mon, 10 Feb 2020 23:21:13 GMT

Thanks for the response to4kawa,

I apologize, I don't think I made my issue clear from the earlier example. I need "Connection: misc" to not appear no matter where it shows up in the data. Using this for the raw instead:

Connection: lost
Connection: finding
Connection: found
Unneeded data
Connection: misc.
Connection: misc.
unneeded data
Connection: lost
Connection: finding
Connection: found
Unneeded Data
Connection: lost
Connection: finding
Connection: still finding"

Your search returns 1-3 and 5-6 with session as 1, 8-10 with session as 2, and 12-14 with session as 3. My transaction example returns the same as well just because keepevicted seems to keep things very general when it is used. In this case, I would only want 1-3, 8-10, and 12-14

Re: Guarantee startswith when using keepevicted

to4kawa — Wed, 12 Feb 2020 12:12:55 GMT

| makeresults 
| eval _raw="Connection: lost
Connection: finding
Connection: found
Unneeded data
Connection: misc.
Connection: misc.
unneeded data
Connection: lost
Connection: finding
Connection: found
Unneeded Data
Connection: lost
Connection: finding
Connection: still finding" 
| makemv delim="
" _raw 
| rename _raw as raw 
| mvexpand raw 
| rename raw as _raw
    `comment("this is your sample, from here, the logic")` 
| search "Connection" NOT "misc"
| streamstats count(eval(searchmatch("lost"))) as session 
| where session > 0