topic Re: How can I do a stable sort? in Splunk Search

How can I do a stable sort?

jdjdjdjd — Fri, 04 Mar 2016 20:57:16 GMT

I am trying to create a view that merges log records from various files, ordered by their timestamps. This works nicely, except when there are entries with the same timestamp. Can Splunk do a stable sort?

From https://en.wikipedia.org/wiki/Category:Stable_sorts:

Stable sorting algorithms maintain the relative order of records with equal keys (i.e. values). That is, a sorting algorithm is stable if whenever there are two records R and S with the same key and with R appearing before S in the original list, R will appear before S in the sorted list.

Re: How can I do a stable sort?

somesoni2 — Fri, 04 Mar 2016 23:15:00 GMT

Can you provide the query that you're currently using?

Re: How can I do a stable sort?

jdjdjdjd — Fri, 04 Mar 2016 23:21:39 GMT

My query looks like this:

index=*mysite* 29f91eb36868446fbf1ae667c895923c | sort _time

Re: How can I do a stable sort?

somesoni2 — Fri, 04 Mar 2016 23:34:47 GMT

Can post examples (just the timestamp) where you think Splunk is not doing a stable sort? IMO, for events with same timestamp, Splunk will keep them in the order they were retrieved by Splunk (non-chronological order).

Re: How can I do a stable sort?

jdjdjdjd — Fri, 04 Mar 2016 23:39:56 GMT

Here's an example. I'm exporting from Splunk in raw format, that's where I'm seeing the problem.

{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",

Re: How can I do a stable sort?

jdjdjdjd — Fri, 04 Mar 2016 23:48:07 GMT

On closer examination, I'm seeing the same results even without sort, so it seems as if Splunk is retrieving my records in the "wrong order" when they have the same timestamp.

Re: How can I do a stable sort?

somesoni2 — Fri, 04 Mar 2016 23:51:57 GMT

So the order Splunk provides for the data with same timestamp is not correct?
I'm not super sure about requirement here, but my guess will that you want to events in increasing order of _time, where Splunk shows events in decreasing order of _time. If you just want to reverse the order, Splunk provides a command reverse, that will do exactly the same.

index=*mysite* 29f91eb36868446fbf1ae667c895923c | reverse

If that's not what you want, try this dirty workaround

 index=*mysite* 29f91eb36868446fbf1ae667c895923c | streamstats count as rank by _time | sort _time -rank | fields - rank

Re: How can I do a stable sort?

jdjdjdjd — Sat, 05 Mar 2016 00:11:51 GMT

You are a wizard! The dirty workaround looks like the answer. Can you post this as an answer rather than a comment?

Is there a way to encapsulate this so that I don't have to copy and paste it each time?

Re: How can I do a stable sort?

somesoni2 — Sat, 05 Mar 2016 01:17:07 GMT

Need to test it but try to put the string as macro.