Splunk Search

Sort command causing lost records?

buttsurfer
Path Finder

I have a very simple search and when I add the sort command i lose almost 90% of my actual results. 

 

 

index="features" application=kokoapp type=userStats
| sort feature
| dedup feature
| table feature 

 

 

 

Without the sort command I get 35 results and with it included i only get 4 results. Is there something I am missing?

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @buttsurfer,

sort cuts results only if you have more than 10,000 results.

probably is dedup that deletes some events.

Ciao.

Giuseppe

0 Karma

buttsurfer
Path Finder

Turns out there actually are over 10,000 results. But I need them all - is there a workaround to this?

0 Karma

yuanliu
SplunkTrust
SplunkTrust

You can try to raise limit in limits.conf.  But I don't understand what you mean by you need all >10,000 results because your dedup without sort only gives you 35.

If you want to sort by feature, sort after dedup. (Sort is memory-intensive; performing sort on any sizable chunk should always be deliberate.)

index="features" application=kokoapp type=userStats
| dedup feature
| sort feature
| table feature 

This should give you the same desired result.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Definitely you should 1st dedup and then sort. Otherwise there is possibility that you hit 10k limits and even that you wasted resources.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

10k limit for sort is one thing but if you say that you get 4 out of 35 results, that's way under 10k 😉

Remember that dedup works by leaving first event with given field(s) and then dropping other subsequent results with this field(s). So it is most probably the culprit here. When you sort by a field then dedup by this field you'll effectively leave just one event per each possible value of this field.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @buttsurfer,

as you can read at https://docs.splunk.com/Documentation/SplunkCloud/latest/SearchReference/Sort yu could use:

| sort 0 feature

in this way you don't limit results.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | What did the zero say to the eight?

June 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

Splunk Observability Cloud's AI Assistant in Action Series: Onboarding New Hires & ...

This is the fifth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Now Playing: Splunk Education Summer Learning Premieres

It’s premiere season, and Splunk Education is rolling out new releases you won’t want to miss. Whether you’re ...