Getting Data In

Why are the number of events indexed in Splunk less than the number of events in the monitored log file?


We are monitoring a file name X.log witch contain similar structure for events starting by a date format. The number of events that were indexed in Splunk are less than the number of events in the file. Is there any bug in indexing ? what logs should i look at ?

0 Karma

Esteemed Legend

It is also possible that your timestamping is off and some of your events are showing up way at the wrong time (even in the future). Try this search run for ALL TIME (you have to use "ALL TIME" because that is the only way to search for events "in the future):

index=myIndex sourcetype=mySourceType | eval bytes=len(_raw) | stats count sum(bytes) by source

This will allow you to compare against your sourcefiles with this linux CLI:

cat myFile | wc

If the bytes match, then all of your file is going into Splunk. If the "count" equals "lines" then all the lines are broken correctly.

0 Karma


If your events are single line it is possible that Splunk has combined some of them accidentally. You can configure event line breaking by following the guidance here -->

In your case you may be most interested in the BREAK_ONLY_BEFORE parameter.

Get Updates on the Splunk Community!

There's No Place Like Chrome and the Splunk Platform

Watch On DemandMalware. Risky Extensions. Data Exfiltration. End-users are increasingly reliant on browsers to ...

The Great Resilience Quest: 5th Leaderboard Update

The fifth leaderboard update for The Great Resilience Quest is out >> 🏆 Check out the ...

Devesh Logendran, Splunk, and the Singapore Cyber Conquest

At this year’s Splunk University, I had the privilege of chatting with Devesh Logendran, one of the winners in ...