Splunk Search

Natural sort order instead of lexicographical order in search?

splunknoob408
Explorer

I've got a date field that I extracted from log messages, and it is pulled from two different sources. One source zero-pads the numbers, so I get dates like 12/08/17, while the other does not and gives me 12/8/17.

The problem is during sort, where I want a natural sort order, but the lexicographical sorting swaps the date entries around.

Is there a way in splunk to force it to zero-pad the dates so they are sorted properly? I have seen references to using a strptime() function, but don't know how to use it in my particular example. Here is my search, which works, except that the X axis (date) has its columns swapped around due to the lexicographical sorting:

source=OrderEventsTable service="tracking" event_details="shipped on*" | regex event_details="\d+/\d+/\d+" | dedup order_id | stats count as "shipped number" by shipped_date

I only know that I would want to pass strptime %m/%d/%y, since %d zero-pads the day. I tried this, unsuccessfully, because it changed shipped_date into a decimal number.

source=OrderEventsTable service="tracking" event_details="shipped on*" | regex event_details="\d+/\d+/\d+" | dedup order_id | stats count as "shipped number" by shipped_date | eval shipped_date=strptime(shipped_date,"%m/%d/%y")
0 Karma
1 Solution

MonkeyK
Builder

splunknoob408,
When sorting by time, epoch time (decimal number version) works best. You can convert to a human readable format after.

Try this:

source=OrderEventsTable service="tracking" event_details="shipped on*" 
| regex event_details="\d+/\d+/\d+" 
| dedup order_id 
| eval shipped_date=strptime(shipped_date,"%m/%d/%y") 
| stats count as "shipped number" by shipped_date 
| sort shipped_date
| eval shipped_date=strftime(shipped_date,"%m/%d/%y") 

View solution in original post

MonkeyK
Builder

splunknoob408,
When sorting by time, epoch time (decimal number version) works best. You can convert to a human readable format after.

Try this:

source=OrderEventsTable service="tracking" event_details="shipped on*" 
| regex event_details="\d+/\d+/\d+" 
| dedup order_id 
| eval shipped_date=strptime(shipped_date,"%m/%d/%y") 
| stats count as "shipped number" by shipped_date 
| sort shipped_date
| eval shipped_date=strftime(shipped_date,"%m/%d/%y") 

splunknoob408
Explorer

Awesome answer, thank you so much for the help! This totally worked.

0 Karma

niketn
Legend

@MonkeyK, after sorting epoch date, I would use fieldformat instead of the last eval. So that field remains epoch date and value converts to string date.

 | fieldformat shipped_date=strftime(shipped_date,"%m/%d/%y") 
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

MonkeyK
Builder

a new command! thank you! This will be useful in a number of my existing queries

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...