We tried this search below:
index=test | eval dup=_raw | convert ctime(_time) as T1 | transaction dup mvlist=t maxspan=1s keepevicted=true | where eventcount>1
And these are sample events:
_time : 4/9/2015 12:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
_time : 4/9/2015 12:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
_time : 4/9/2015 12:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
time : 4/9/2015 13:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
time : 4/9/2015 13:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
time : 4/9/2015 14:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
time : 4/9/2015 15:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
I want latest events like the example below, remove the other events without using dedup command,
time : 4/9/2015 15:00:01 :-----host:foo1, source:foo2, source type:foo3, Date:foo4,status=foo5,Name=foo6.
Another strategy for this one, faster than dedup if the same fields are consistently present:
index=test
| stats latest(_time) as _time latest(_raw) as _raw by host source sourcetype Date status Name
...or ...
index=test
| stats latest(_time) as _time by _raw
...but the second one means you need to re-extract the other fields.
Why do you want to avoid using dedup? Dedup is by far the easiest way of doing this. Just |dedup host, source, sourcetype, Date, Name | , and that's all you need to do. Add that before the transaction.
Dedup will leave you with the most recent result from each duplicate set.
From the docs: "Events returned by dedup will be the first event found in search order, which is the most recent for historical search"