Getting Data In

Why are we having an indexing issue in folder monitoring from syslog server?

evelenke
Contributor

Hi Splunkers,

Please help in resolving the following issue.
We have a lot of folder monitoring from syslog server. Each folder contains logs from some of device divided by dates.
One of this logs may have long term pauses in indexing - up to 5 hours. After it starts indexing again the data for previous period is never updated so we have gaps.

alt text

File size is on average 3-4 Gb daily.

In logs I see INFO messages with information that the file has been read:

metrics.log
03-29-2018 14:44:50.336 +0300 INFO  Metrics - group=per_host_thruput, ingest_pipe=1, series="**myhost**", kbps=57.65211637507532, eps=353.7979033990437, kb=1787.2587890625, ev=10968, avg_age=157705281.7769876, max_age=157762827

splunkd.log
03-29-2018 13:37:20.064 +0300 INFO  TailReader - Batch input finished reading file='/..path/**myhost**/2018/2018-03/2018-03-29/2018-03-29_**myhost**.txt'
0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...