Splunk Search

Real time searches with future time

phoenixdigital
Builder

We are currently indexing data which contains predicted values for data into the future.

I am having trouble working out how to have a real time search which will chart these 'future' predictions as they arrive.

I can chart this data perfectly as a normal search showing the chart into the future. However when I try to have this as a chart based on real time events it will only show up until now.

As the prediction data comes in we are assigning the 'prediction time' to _time.

However if I add earliest time as rt-60m and the latest time as rt+60m. Looking at the results Splunk only appears to pull the data in up to now. Not into the future as requested in 'latest time'.

Does anyone have any workarounds without me needing to convert the 'prediction time' into another field? or worse still messing with charts to handle custom times rather than _time.

Tags (1)
0 Karma
1 Solution

phoenixdigital
Builder

Thanks for the suggestion araitz. Unfortunately showing all time in a chart would be too much data where I only need 60 minutes ago and 60 minutes into the future.

Acutally it appears to work ok in my advanced dashboard with

      earliest rt-60m
      latest rt+60m

I think the issue was occurring due to me being in a daylight savings state when testing and the server was set to a non daylight savings time. (Grrrr DST)

I have just tested again now that I am back in a real timezone and it appears to work as expected.

Apologies. Keep up the good work Splunk.

On a side note I spoke to a Splunk engineer yesterday and he said there was a setting in Splunk to automatically junk any data coming in that has a _time of more than 48 hours into the future.

He said that this could be disabled in one of the config files should the need arise. Seeing as I only have data a few hours into the future this is not a concern. I just thought I would mention it here if it helps someone in the future.

View solution in original post

0 Karma

phoenixdigital
Builder

Thanks for the suggestion araitz. Unfortunately showing all time in a chart would be too much data where I only need 60 minutes ago and 60 minutes into the future.

Acutally it appears to work ok in my advanced dashboard with

      earliest rt-60m
      latest rt+60m

I think the issue was occurring due to me being in a daylight savings state when testing and the server was set to a non daylight savings time. (Grrrr DST)

I have just tested again now that I am back in a real timezone and it appears to work as expected.

Apologies. Keep up the good work Splunk.

On a side note I spoke to a Splunk engineer yesterday and he said there was a setting in Splunk to automatically junk any data coming in that has a _time of more than 48 hours into the future.

He said that this could be disabled in one of the config files should the need arise. Seeing as I only have data a few hours into the future this is not a concern. I just thought I would mention it here if it helps someone in the future.

0 Karma

araitz
Splunk Employee
Splunk Employee

Using a real-time window of all-time should allow you to see all events, even those in the "future".

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...