Splunk Search

Is there any way to use the date_month field in a search as a numeric field without performance loss?

Riel
Engager

Hi,

As you know, date_month, date_mday, date_year fields are so useful to fetch data quickly.
I usually use these fields in my main search, but since the value of the date_month field also contains alphanumeric (a-z) characters, not just numbers, I always face problems using it in a search.
Sure, there are a lot of ways to convert it to numeric such as using lookup table or strftime eval function, but all of these ways lose performance to fetch data at least 2~4 times.

I really want to know a way to use numeric date_month field with minimize to lose data fetch performance.

1 Solution

woodcock
Esteemed Legend

You may not realize this, but although the date* fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month field is "correct" is when date_zone=0 or date_zone=local (and most of the time, when date_zone=local the timestamp itself is wrong because somebody forgot to put in a TZ= into that props.conf). For this reason, I rarely use them and create my own like this:

| eval month=strftime(_time,"%m")

For more info, see here:

http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html

View solution in original post

woodcock
Esteemed Legend

You may not realize this, but although the date* fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month field is "correct" is when date_zone=0 or date_zone=local (and most of the time, when date_zone=local the timestamp itself is wrong because somebody forgot to put in a TZ= into that props.conf). For this reason, I rarely use them and create my own like this:

| eval month=strftime(_time,"%m")

For more info, see here:

http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html

Riel
Engager

I think using eval function has a little bit loss of fetching data performance.
I'll compare the ways between using eval function and using props.conf.
Thanks all.

0 Karma

woodcock
Esteemed Legend

You can use the job inspector to evaluate and compare.

0 Karma

somesoni2
Revered Legend

You can save this "Month" field as calculated field in your props.conf and get better performance.

Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...