Dedup lost data, a bug?

[april_tao]

For below search :

eventtype=MYTYPE [search eventtype=MYTYPE | sort 0 _time desc | dedup fieldX | return 1000 source]

Expect to return the latest source for 1 fieldX value.

In our data, we have over 10,000,000 events for the latest source with fieldX=A, fieldX=B, fieldX=C respectively.
Thus expect the search returns over 30,000,000 results.
However, it returns the results for fieldX=A only.

Question : is the search correctly written? If yes, is this a bug of dedup? Is there any limitation of dedup about the result size? If we use a smaller dataset, dedup works properly with the same search.

[lguinn2]

The problem is that the subsearch has a limit - and I don't see that you need the subsearch at all. You also do not need the sort, Splunk returns events in reverse time order (newest first) by default.Try this

eventtype=MYTYPE | dedup fieldX

This will return the most recent event for each value of fieldX.