Getting Data In

Why are we having an indexing issue in folder monitoring from syslog server?

evelenke
Contributor

Hi Splunkers,

Please help in resolving the following issue.
We have a lot of folder monitoring from syslog server. Each folder contains logs from some of device divided by dates.
One of this logs may have long term pauses in indexing - up to 5 hours. After it starts indexing again the data for previous period is never updated so we have gaps.

alt text

File size is on average 3-4 Gb daily.

In logs I see INFO messages with information that the file has been read:

metrics.log
03-29-2018 14:44:50.336 +0300 INFO  Metrics - group=per_host_thruput, ingest_pipe=1, series="**myhost**", kbps=57.65211637507532, eps=353.7979033990437, kb=1787.2587890625, ev=10968, avg_age=157705281.7769876, max_age=157762827

splunkd.log
03-29-2018 13:37:20.064 +0300 INFO  TailReader - Batch input finished reading file='/..path/**myhost**/2018/2018-03/2018-03-29/2018-03-29_**myhost**.txt'
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...