Can I add events to a transaction?

[MonkeyK]

I have a transaction based on a bunch of events from a common source with a common transaction ID, something like

|"search" | transaction by tid

This will get me results like

2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:31,502 tid:10001
-----------------------------------------------------
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:11,502 tid:10000
-----------------------------------------------------

but I need to associate the transaction events with an event from another source. that source has an outcome that happens within 1s of the transaction in the first source.

2017-04-11 04:20:33,502 src_ip=192.168.1.99 result=success
2017-04-11 04:10:12,502 src_ip=192.168.1.15 result=fail

Is there a way to add events to my transactions or otherwise join based on the last time in the transactions? Ideally I'd like to be able to see results like:

-----------------------------------------------------
2017-04-11 04:20:33,502 src_ip=192.168.1.99 result=success
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:31,502 tid:10001
-----------------------------------------------------
2017-04-11 04:10:12,502 src_ip=192.168.1.15 result=fail
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:11,502 tid:10000
-----------------------------------------------------

[somesoni2 · Accepted solution]

I guess you can try this

"search" | transaction by tid
| append ["search for source 2" ]
| transaction maxspan=1s

[woodcock · Accepted solution]

Try this:

 "search for source 1" | transaction BY tid
| eval times=_time
| append [ "search for source 2" | eval times = (_time-1) . " " . _time . " " . (_time+1) | makemv times ]
| transaction times maxspan=1s

[MonkeyK]

Woodcock, this is a really neat idea. I think that it doesn't work for me, because it counts on times aligning to the second. To be fair, my example only showed 1s precision.

I think that somesoni2's answer is very close to what I need but that I may be running into some kind of result set limit. I think this because I get results for 1 day, but if I expand my query to a month, the 1 day results disappear.

But with some tweaking, your idea may solve a more general problem of joining datasets by time with sub-second proximity precision (which transaction does not do).
This can be important when we need to correlate datasets but there is more than 1 set of events per second.

I had been trying to solve this by using "bin _time" but the problem is that we cannot control which bin the event lands in. By adding values and comparing the additional values, I can be much more certain to land on a matchable time

for subsecond transactions on 1 source
"search for source 1" | bin _time span=200ms | eval times=_time . " " . (_time+0.2) | makemv times
| transaction times

and on two
"search for source 1" | transaction BY tid
| bin _time span=200ms | eval times=_time . " " . (_time+0.2) | makemv times
| append [ "search for source 2" | bin _time span=200ms | eval times=_time . " " . (_time+0.2) ]
| transaction times

To be fair, the bin could result in events up to 400ms apart, but this is still way better than a precision of 1s
I did run into this problem with some of my results, but it was rare enough that I figured I could just call out the discrepancy in the results

Splunk could solve the same problem by allowing smaller transaction span units

[somesoni2 · Accepted solution]

I guess you can try this

"search" | transaction by tid
| append ["search for source 2" ]
| transaction maxspan=1s

[MonkeyK]

Just tried it. The second transaction does not get combined with the first and I get all events from the appended search, so it ends up looking like

2017-04-11 04:21:33,502 src_ip=192.168.1.97 result=success

---

2017-04-11 04:20:36,502 src_ip=192.168.1.98 result=success

---

2017-04-11 04:20:33,502 src_ip=192.168.1.99 result=success

---

2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:32,502 tid:10001
2017-04-11 04:20:31,502 tid:10001

---

2017-04-11 04:15:02,502 src_ip=192.168.1.20 result=fail

---

2017-04-11 04:14:52,502 src_ip=192.168.1.19 result=fail

---

2017-04-11 04:13:42,502 src_ip=192.168.1.18 result=fail

---

2017-04-11 04:12:32,502 src_ip=192.168.1.17 result=fail

---

2017-04-11 04:11:22,502 src_ip=192.168.1.16 result=success

---

2017-04-11 04:10:12,502 src_ip=192.168.1.15 result=fail

---

2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:12,502 tid:10000
2017-04-11 04:10:11,502 tid:10000

[MonkeyK]

Whoa. I can actually almost make this work if I sort before the second transaction:
"search" | transaction by tid
| append ["search for source 2" ]
|sort 0 _time
| transaction maxspan=1s

That's what I get for over-sanitizing the question and results sets

[MonkeyK]

What I cannot figure out is why I get 1 transaction record back when I search on 3/15,
but if i search between 3/15-4/5 I get 61 transactions records back with none on 3/15.

I must be running into some some kind of limit

[woodcock]

This is why I almost never use transaction; it has silently-enforced limits that are exceedingly easy to hit. This is one of the reasons many people say "get as much RAM as you can", even 10X of suggested amounts.

[MonkeyK]

Wow. Very useful advice! I will rethink my query and see what I can do without transactions.