Getting Data In

Why are we having an indexing issue in folder monitoring from syslog server?

evelenke
Contributor

Hi Splunkers,

Please help in resolving the following issue.
We have a lot of folder monitoring from syslog server. Each folder contains logs from some of device divided by dates.
One of this logs may have long term pauses in indexing - up to 5 hours. After it starts indexing again the data for previous period is never updated so we have gaps.

alt text

File size is on average 3-4 Gb daily.

In logs I see INFO messages with information that the file has been read:

metrics.log
03-29-2018 14:44:50.336 +0300 INFO  Metrics - group=per_host_thruput, ingest_pipe=1, series="**myhost**", kbps=57.65211637507532, eps=353.7979033990437, kb=1787.2587890625, ev=10968, avg_age=157705281.7769876, max_age=157762827

splunkd.log
03-29-2018 13:37:20.064 +0300 INFO  TailReader - Batch input finished reading file='/..path/**myhost**/2018/2018-03/2018-03-29/2018-03-29_**myhost**.txt'
0 Karma
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...