topic Re: future events on timechart in Dashboards & Visualizations

future events on timechart

s8ea8n — Thu, 07 Jun 2012 17:28:57 GMT

I am trying to index data for future events (reservation searches), to (line)chart when people are looking to travel. These dates range from a year ago to a few years in the future. It seems to be giving future events a timestamp of 3 days from the time the data was loaded.

Is there any trick to this?

Re: future events on timechart

sideview — Mon, 28 Sep 2020 11:54:54 GMT

It's the default setting of a key called MAX_DAYS_HENCE. Splunk is hitting up against a failsafe here. You should weaken it just for that particular sourcetype.

here are the docs:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

Re: future events on timechart

DrewO — Mon, 28 Sep 2020 11:54:57 GMT

So it looks like there is a timestamp confusion here. The timestamp that Splunk creates when it indexes data is ideally drawn from the time the event was written to the log file/database/etc. In your use case here it seems that date fields you are working with are separate from the timestamp.

So to make up an example:

June 7, 2012 15:37 EDT Requested reservation SFO - LAX out: Sept 1 2012 return: Sept 10 2012

In my made up example here "June 7 2012 15:37 EDT" would be the timestamp. It's the time and date i attempted to make the reservation. The subsequent dates Sept 1 and Sept 10 are not the timestamp, they are just date based fields contained in the data. You want to run reports on the date fields not the timestamp.

Not knowing exactly what your data looks like I can't give you specifics on how to let Splunk know the difference between a date field and the actual timestamp of the event. BUT I can point you to this doc page:

http://docs.splunk.com/Documentation/Splunk/4.3.2/Data/HowSplunkextractstimestamps

For your particular use case it may make sense to disable timestamping all together:

http://docs.splunk.com/Documentation/Splunk/4.3.2/Data/TuneTimestampExtractionForBetterIndexingPerformance#Disable_timestamp_processor

Then you can extract out the reservation dates as separate fields and run your reports against those fields.

http://docs.splunk.com/Documentation/Splunk/4.3.2/Knowledge/Addfieldsatsearchtime

Also the reason no dates beyond 3 days in the future would index is because Splunk is set to automatically ignore future timestamps beyond 3 days.

See the Max_Days_Hence information here: http://docs.splunk.com/Documentation/Splunk/latest/admin/propsconf

If you are using Splunk 4.3 the Preview feature in Manager can be VERY helpful when you need to teach Splunk where to look and where NOT to look for a timestamp.

Re: future events on timechart

s8ea8n — Mon, 28 Sep 2020 11:55:16 GMT

Thanks. Disabling timestamping did the trick and the other references helped with understanding why. Below is my working search after making the change.

index=CALENDAR_SEARCHES | convert timeformat="%Y-%m-%d %T" mktime(CALENDAR_DATE) | fieldformat CALENDAR_DATE=strftime(CALENDAR_DATE,"%Y-%m-%d %T") | makecontinuous CALENDAR_DATE span=1m | chart avg(COUNT) by CALENDAR_DATE

Re: future events on timechart

sideview — Sat, 09 Jun 2012 02:41:31 GMT

I guess I was assuming that these timestamps were the "correct" times that you wanted the events indexed at. 😃 Sounds like that was not the case.