Archive

In a Splunk dashboard, how do you track data ingestion within a certain time?

New Member

Hello,

On a Splunk dashboard, Is there a way to show when data was ingested, stored, and analyzed? I'm trying to build a dashboard to show the following:

  1. Source time
  2. Arrival time for data
  3. Fetch time when it was available
  4. ingested time

I want to prove that it all happened within 10 minutes, and then show that proof in a dashboard. I'm using Splunk Enterprise Search and reporting.

Could you please help me write a query for that or if there are any examples that would be great. Thank you for help in advance!

0 Karma
1 Solution

Influencer

Source time is your _time field in the index
Ingested time/arrival time/fetched time all should be same that is when the data got ingested it is available for search- the field is _indextime

View solution in original post

Influencer

Source time is your _time field in the index
Ingested time/arrival time/fetched time all should be same that is when the data got ingested it is available for search- the field is _indextime

View solution in original post

New Member

This is what i have so far, however, the time is in seconds now, and i don't understand what that time means. Is there a way to change the seconds to (H:M:S). Again thank you for help!

Source = " " index= " " host = " "
| eval delay_sec=_indextime-_time
| timechart min(delay_sec) avg(delay_sec) max(delay_sec) by source

0 Karma

Influencer

The value delay_sec should be in seconds , you don't need to convert it in H:M:S format. If you want to display in minutes, you can divide it by 60

0 Karma

New Member

I tried dividing by 60 it didn't show any results.

0 Karma

Influencer

what is the value it shows you before dividing by 60?

0 Karma

Influencer

Ideally you should be getting the delay in seconds and you do not need any conversion if the delay is usually a few seconds.

0 Karma

New Member

Here are all the values i'm getting after running this query:
Source = " " index= " " host = " "
| eval delay_sec=_indextime-_time
| timechart min(delay_sec) avg(delay_sec) max(delay_sec) by source

avg(delay_sec): 19720226
max(delay_sec): 19936226
min(delay_sec): 19504226

I'm trying to check how long did this process takes after ingested data, and when it shows on my search by having a dashboard. But the values above is not really telling me much. Is there a better way ? Sorry for asking a lot of questions

0 Karma

Influencer

Use the query like this-

Source = " " index= " " host = " " 
| eval delay_sec=_indextime-_time 
| timechart min(delay_sec)  as min_delay avg(delay_sec) as avg_delay max(delay_sec) as max_delay by source
| eval avg_delay=strftime(avg_delay, "%Y-%m-%d %H:%M:%S")
| eval min_delay=strftime(avg_delay, "%Y-%m-%d %H:%M:%S")
| eval max_delay=strftime(avg_delay, "%Y-%m-%d %H:%M:%S")
0 Karma

New Member

I got the same exact values in seconds still

0 Karma

Influencer

You actually dont need to use strftime. The delay is the timedifference in seconds between the index time and the sourcetime.
By looking at the values of result it looks like there is a huge delay as 19720226 seconds which is around 350 days. Can you share the value of _time from your logs

0 Karma

New Member

_time: 2018-01 I'm testing old data but this is just for testing purpose and learning. I'm just trying to see if it's possible to do it the way i have it already.
Also, min, avg, max what does those exactly tell me in this case?

0 Karma

Influencer

Well min here tells you minimum difference between indextime and _time for a particular source, avg will give you an average delay, and max will give you maximum delay between index time and _time field for a given source.
Also the value of _time depends on the datetimestamp configured for your logs. Ideally it should be date timestamp inside your logs but sometimes it is set as Current date and time in which case index time and _time would be same.
The best way to check the different between a set of ecents would be just to display logs with _time and _indextime value

index = <yourindex> sourcetype=<yoursourcetype>| eval time=strftime(_time,"%Y-%m-%d %H:%M:%S")| eval  indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S")| table _raw time indextime
0 Karma

New Member

Thank you!

0 Karma