topic Why is the time picker not working with date in logs? in Splunk Search

Why is the time picker not working with date in logs?

davidcraven02 — Thu, 05 Apr 2018 14:26:35 GMT

Data is forwarded to Splunk every couple of days meaning that the _time stamp relates to the day it was sent to Splunk. The actual date of the event is listed in the logs as e.g Date="2018-03-29 11:48".

How can I make the time picker search the date of the event (Date) rather than the date (_time) the data was ingested by Splunk?

index=summary report=jiracsatresults Key="**" Assignee="**" Classification="**"
| dedup Key
| eval dateEpoch = strptime(Date, "%Y-%m-%d %H:%M")
| eval today = now()
| eval daysAgo = round(((today - dateEpoch)/60/60/24), 0) 
|  search daysAgo <= 7
|  table Key, Summary, Reporter, Assignee, Classification, "CSAT Rate", "CSAT Rating Comment", Date

The only way I get the results for a specific Date e.g for a week is using daysAgo <= 7

Re: Why is the time picker not working with date in logs?

skoelpin — Thu, 05 Apr 2018 14:46:28 GMT

You need to configure props.conf on the indexer(s) to read the timestamp in the log file and assign it while indexing. If Splunk can't determine a timestamp, it will assign a timestamp of when the event was indexed. Post some sample data and I will help you create that props.conf stanza

Re: Why is the time picker not working with date in logs?

davidcraven02 — Tue, 29 Sep 2020 18:56:15 GMT

Ok perfect thanks. Here is an example raw log.

04/02/2018 09:00:00 +0100, search_name="Support - CSAT Results", search_now=1522662477.000, info_min_time=1522656000.000, info_max_time=1522662477.000, info_search_time=1522662477.324, Assignee=test_user, CSAT Rate=5, Classification="Hardware Issues - PC", Date="2018-03-29 11:48", Key="TEST-457", Reporter=test_user, Summary="Laptop Health Checks", report="jiracsatresults"

Re: Why is the time picker not working with date in logs?

skoelpin — Thu, 05 Apr 2018 15:14:06 GMT

Add this stanza to your props.conf and restart the splunkd service on the indexers and it will work correctly

[stash]
TIME_PREFIX = Date="
TIME_FORMAT = %Y-%m-%d %H:%M
MAX_TIMESTAMP_LOOKAHEAD = 18

Re: Why is the time picker not working with date in logs?

davidcraven02 — Thu, 05 Apr 2018 18:11:03 GMT

Thank you but I have just realized that will affect other other logs that actually have the correct date format.

Would it be possible to have two text inputs "Earliest" and "Latest" which a user can enter the number of days for both to select a period of time. Not sure how to actually implement that.

i.e "Earliest" = 40 days ago AND "Latest" = 20 days ago

Re: Why is the time picker not working with date in logs?

skoelpin — Thu, 05 Apr 2018 18:23:28 GMT

This is the importance of sourcetypes.. You assign a sourcetype on the "shape" of the data. Since this data format has a different shape, you need to assign it a different sourcetype.

The answer to your question is yes, but the best approach would be to assign it a new sourcetype.

You will need to first convert that date format into epoch time, then specify the earliest and latest

Re: Why is the time picker not working with date in logs?

skoelpin — Fri, 06 Apr 2018 00:07:13 GMT

Please accept the answer if this answered your question