Summary
I have a common field shared between two events which is a phone number. One event has details about the type of phone model and the other event is a RADIUS accounting start message for the same phone number, these two events occur within seconds of each other.
| transaction phonenum maxpause=5s
I then want to take these two events and combine them with | transaction which I have done without issue, my problem is I am trying to do a subsequent search to include the accounting stop message along with the first transactions output. Since the original transaction I executed also includes some additional fields, specifically a session_id field I can ensure that the correct stop message is associated with it when it decides to show up.
So basically there are three events in total, but the third event could take hours or days to show up and I don't simply want to do a | transaction over days due to CPU consumption time.
I have spent a lot of time searching and reading to find the best approach, but haven't really found anything that speaks to my scenario.
My initial thought was to run the first transaction and store the fields I care about into a summary index every 15 mins and then run a second search which tries to | transaction the new combined event with the last event (accounting stop message), however again as the summary index grows and the stop messages potentially taking days to arrive this could lead to 30day+ searches to correlate the full transaction.
My next thought was to run the first | transaction and store it into an outputlookup and then do a periodic lookup against this table for just the accounting stop messages which would filter out the sessions that had truly stopped, but since my outputlookup contains a few fields, session_id, handset_model etc since only the session_id matches the accounting stop message the handset_model field which was present in the original outputlookup csv that field isn't retained in the found events.
I then took this same approach but ran it with | join and managed to get a single event with both the stop message and the handset model present, but I read all over answers that | join is not a good choice to use so I am now questioning it's use.
So I guess I am looking for some pointers on the best way to tackle this issue, my end goal is to feed a new event that includes a customers start, stop and handset model so I can easily report data usage by handset over long periods of time like 30 days for example and not cripple the box. It would seem I need a combination of scheduled searches with the final cooked event sent to a summary index for reporting purposes.
Would someone be so kind as to give me their 2 cents
Thanks
Jerrad
... View more