Getting Data In

How to merge closest events by time for each host?

Path Finder

Hello!

I have a lot of different events of such kind:
host="A" datetime="time1.1" otherfield="fld"
host="A" datetime="time1.2" other
field="fld"
host="A" datetime="time1.3" otherfield="fld"
host="B" datetime="time2.1" other
field="fld"
host="B" datetime="time2.1" otherfield="fld"
host="A" datetime="time3.1" other
field="fld"
host="A" datetime="time3.2" otherfield="fld"
host="A" datetime="time4.1" other
field="fld"
host="A" datetime="time4.2" otherfield="fld"
host="A" datetime="time4.3" other
field="fld"

And want to report it by email like this:
host="A" datetime="time1" otherfield="fld"
host="B" datetime="time2" other
field="fld"
host="A" datetime="time3" otherfield="fld"
host="A" datetime="time4" other
field="fld"

So I want to merge closest events (by time) for one host. What is the right way to do it?

Thanks!

Tags (3)
1 Solution

SplunkTrust
SplunkTrust

Give this a try

your base search | bucket span=1s _time | dedup _time,host,other_field

View solution in original post

SplunkTrust
SplunkTrust

Give this a try

your base search | bucket span=1s _time | dedup _time,host,other_field

View solution in original post

SplunkTrust
SplunkTrust

What is the difference in the time sets (1.1, 1.2, 1.3 or 3.1, 3.2, etc.)? Are you grouping them because they are within a certain range, or is there some other criteria that you are using?

0 Karma

Path Finder

It depends. But in fact it is a certain range. I mean there are several events on one host in one second, so time{1.1, 1.2, 1.3} are different milliseconds in one second. Or you can think about these values as sequence of time values in some time gap. I used transactions to merges events in this way

sourcetype='a_lot_of_events_in_little_time' other_field="fld" | transaction host maxspan=2s

But it is not the result I want, because transaction has multiple records inside itself. I need only one in time range.

Imagine one file modification creates a lot of events (for example 6 in one second) and sends them to logging system. But you interested only in the fact of file modification. So you want to merge these events into one and alert about it. But you don't want to miss next file modification that can happen right after the first one.

0 Karma

You should be able to use timechart for this. Is datetime equivalent to _time? Then you can do something like:

... | timechart span=2s earliest(_time) values(other_field) by host

Greetings

Tom

0 Karma

Path Finder

Thank you, but the result is not what I need.
Yes, datetime equivalent to _time. Here is an example: datetime="2014-12-04T16:23:29.705893Z".

0 Karma