Splunk Search

Is there any way to use the date_month field in a search as a numeric field without performance loss?

Riel
Engager

Hi,

As you know, date_month, date_mday, date_year fields are so useful to fetch data quickly.
I usually use these fields in my main search, but since the value of the date_month field also contains alphanumeric (a-z) characters, not just numbers, I always face problems using it in a search.
Sure, there are a lot of ways to convert it to numeric such as using lookup table or strftime eval function, but all of these ways lose performance to fetch data at least 2~4 times.

I really want to know a way to use numeric date_month field with minimize to lose data fetch performance.

1 Solution

woodcock
Esteemed Legend

You may not realize this, but although the date* fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month field is "correct" is when date_zone=0 or date_zone=local (and most of the time, when date_zone=local the timestamp itself is wrong because somebody forgot to put in a TZ= into that props.conf). For this reason, I rarely use them and create my own like this:

| eval month=strftime(_time,"%m")

For more info, see here:

http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html

View solution in original post

woodcock
Esteemed Legend

You may not realize this, but although the date* fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month field is "correct" is when date_zone=0 or date_zone=local (and most of the time, when date_zone=local the timestamp itself is wrong because somebody forgot to put in a TZ= into that props.conf). For this reason, I rarely use them and create my own like this:

| eval month=strftime(_time,"%m")

For more info, see here:

http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html

Riel
Engager

I think using eval function has a little bit loss of fetching data performance.
I'll compare the ways between using eval function and using props.conf.
Thanks all.

0 Karma

woodcock
Esteemed Legend

You can use the job inspector to evaluate and compare.

0 Karma

somesoni2
Revered Legend

You can save this "Month" field as calculated field in your props.conf and get better performance.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...