I have logs from two different sources in one search. One source provides a time range, while the other provides a time stamp. I am wondering how these could best be matched (with a minimum of processing power).
Is there any good way to enrich the data like this? The major complication is that the userId and timestamp need to be compared with the userId and timeframe of another batch. The only rather bad way which comes to mind is breaking up the time spans and timestamps into hourly or minutely brackets and then group by them, but that seems to be quite messy.
Thanks. Unfortuntely the amount of data is too big for that case as an inner search will cap out at 50k lines. This search will likely result in a few hundred thousand if not million lines though.
While it would be possible to reduce that, by adding a more compact main search also as subsearch into the join subsearch to limit the total amount, this would result in a search which exceeds the subsearch time limit.
This is why my idea is to go the route of using just two main searches as in
(index=A fields=values) OR (index=B fields=values)
And then trying to merge them together.