Splunk Search

How can I do a stable sort?

jdjdjdjd
Engager

I am trying to create a view that merges log records from various files, ordered by their timestamps.  This works nicely, except when there are entries with the same timestamp. Can Splunk do a stable sort?

From https://en.wikipedia.org/wiki/Category:Stable_sorts:

Stable sorting algorithms maintain the relative order of records with equal keys (i.e. values). That is, a sorting algorithm is stable if whenever there are two records R and S with the same key and with R appearing before S in the original list, R will appear before S in the sorted list.

Tags (1)
0 Karma
1 Solution

somesoni2
Revered Legend

So the order Splunk provides for the data with same timestamp is not correct?
I'm not super sure about requirement here, but my guess will that you want to events in increasing order of _time, where Splunk shows events in decreasing order of _time. If you just want to reverse the order, Splunk provides a command reverse, that will do exactly the same.

index=*mysite* 29f91eb36868446fbf1ae667c895923c | reverse

If that's not what you want, try this dirty workaround

 index=*mysite* 29f91eb36868446fbf1ae667c895923c | streamstats count as rank by _time | sort _time -rank | fields - rank

View solution in original post

0 Karma

somesoni2
Revered Legend

So the order Splunk provides for the data with same timestamp is not correct?
I'm not super sure about requirement here, but my guess will that you want to events in increasing order of _time, where Splunk shows events in decreasing order of _time. If you just want to reverse the order, Splunk provides a command reverse, that will do exactly the same.

index=*mysite* 29f91eb36868446fbf1ae667c895923c | reverse

If that's not what you want, try this dirty workaround

 index=*mysite* 29f91eb36868446fbf1ae667c895923c | streamstats count as rank by _time | sort _time -rank | fields - rank
0 Karma

jdjdjdjd
Engager

You are a wizard! The dirty workaround looks like the answer. Can you post this as an answer rather than a comment?

Is there a way to encapsulate this so that I don't have to copy and paste it each time?

0 Karma

somesoni2
Revered Legend

Need to test it but try to put the string as macro.

0 Karma

somesoni2
Revered Legend

Can you provide the query that you're currently using?

0 Karma

jdjdjdjd
Engager

My query looks like this:

index=*mysite* 29f91eb36868446fbf1ae667c895923c | sort _time

0 Karma

somesoni2
Revered Legend

Can post examples (just the timestamp) where you think Splunk is not doing a stable sort? IMO, for events with same timestamp, Splunk will keep them in the order they were retrieved by Splunk (non-chronological order).

0 Karma

jdjdjdjd
Engager

Here's an example. I'm exporting from Splunk in raw format, that's where I'm seeing the problem.

{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
{"ts":"2016-03-02T17:28:52.461",
0 Karma

jdjdjdjd
Engager

On closer examination, I'm seeing the same results even without sort, so it seems as if Splunk is retrieving my records in the "wrong order" when they have the same timestamp.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...