Getting Data In

Why are we having an indexing issue in folder monitoring from syslog server?


Hi Splunkers,

Please help in resolving the following issue.
We have a lot of folder monitoring from syslog server. Each folder contains logs from some of device divided by dates.
One of this logs may have long term pauses in indexing - up to 5 hours. After it starts indexing again the data for previous period is never updated so we have gaps.

alt text

File size is on average 3-4 Gb daily.

In logs I see INFO messages with information that the file has been read:

03-29-2018 14:44:50.336 +0300 INFO  Metrics - group=per_host_thruput, ingest_pipe=1, series="**myhost**", kbps=57.65211637507532, eps=353.7979033990437, kb=1787.2587890625, ev=10968, avg_age=157705281.7769876, max_age=157762827

03-29-2018 13:37:20.064 +0300 INFO  TailReader - Batch input finished reading file='/..path/**myhost**/2018/2018-03/2018-03-29/2018-03-29_**myhost**.txt'
0 Karma