Splunk Search

How can I do a stable sort?

jdjdjdjd
Engager

I am trying to create a view that merges log records from various files, ordered by their timestamps.  This works nicely, except when there are entries with the same timestamp. Can Splunk do a stable sort?

From https://en.wikipedia.org/wiki/Category:Stable_sorts:

Stable sorting algorithms maintain the relative order of records with equal keys (i.e. values). That is, a sorting algorithm is stable if whenever there are two records R and S with the same key and with R appearing before S in the original list, R will appear before S in the sorted list.

Tags (1)
0 Karma
1 Solution

somesoni2
Revered Legend

So the order Splunk provides for the data with same timestamp is not correct?
I'm not super sure about requirement here, but my guess will that you want to events in increasing order of _time, where Splunk shows events in decreasing order of _time. If you just want to reverse the order, Splunk provides a command reverse, that will do exactly the same.

index=*mysite* 29f91eb36868446fbf1ae667c895923c | reverse

If that's not what you want, try this dirty workaround

 index=*mysite* 29f91eb36868446fbf1ae667c895923c | streamstats count as rank by _time | sort _time -rank | fields - rank

View solution in original post

0 Karma

somesoni2
Revered Legend

So the order Splunk provides for the data with same timestamp is not correct?
I'm not super sure about requirement here, but my guess will that you want to events in increasing order of _time, where Splunk shows events in decreasing order of _time. If you just want to reverse the order, Splunk provides a command reverse, that will do exactly the same.

index=*mysite* 29f91eb36868446fbf1ae667c895923c | reverse

If that's not what you want, try this dirty workaround

 index=*mysite* 29f91eb36868446fbf1ae667c895923c | streamstats count as rank by _time | sort _time -rank | fields - rank
0 Karma

jdjdjdjd
Engager

You are a wizard! The dirty workaround looks like the answer. Can you post this as an answer rather than a comment?

Is there a way to encapsulate this so that I don't have to copy and paste it each time?

0 Karma

somesoni2
Revered Legend

Need to test it but try to put the string as macro.

0 Karma

somesoni2
Revered Legend

Can you provide the query that you're currently using?

0 Karma

jdjdjdjd
Engager

My query looks like this:

index=*mysite* 29f91eb36868446fbf1ae667c895923c | sort _time

0 Karma

somesoni2
Revered Legend

Can post examples (just the timestamp) where you think Splunk is not doing a stable sort? IMO, for events with same timestamp, Splunk will keep them in the order they were retrieved by Splunk (non-chronological order).

0 Karma

jdjdjdjd
Engager

Here's an example. I'm exporting from Splunk in raw format, that's where I'm seeing the problem.

{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
0 Karma

jdjdjdjd
Engager

On closer examination, I'm seeing the same results even without sort, so it seems as if Splunk is retrieving my records in the "wrong order" when they have the same timestamp.

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...