Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Re: Why is splunk not indexing all the data?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why is splunk not indexing all the data?
Hi, lately we've been checking how many files our splunk is indexing, and we noticed that it "skips" some files... We checked by searching:
index=our_index | stats dc(source)
index = _internal group = per_so* series=*.ourfile | stats count
And both ways we got the same results, which are not all the files we indexed.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
A cheerful place to start at I can't find my data!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I didn't find any help in this page....
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also, check internal logs from yoru forwarder(s) to see if there are any warnings/error for your files. ( index=_itnernal sourcetype=splunkd host=yourFwd *filename.ext* )
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've already looked there... Zero warnings or errors....
Also we've tried both monitor and batch input. Both had the same problem...
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try running following on your forwarder instance. See if Splunk is monitoring all the files you've configured for monitoring (will prompt for admin credentials for that Splunk instance)
$SPLUNK_HOME/bin/splunk list monitor
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Even if there are no errors/warnings, do you see any entry for your log file that's missing?
index=_internal sourcetype=splunkd host=yourFwd adding watch
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
well the forwarder doesnt write log when it monitors, only with batch input for some reason. Today we indexed some logs, and we saw one of the files in the forwarders log, but we could not search it...
I wonder if it has something to do with the fact that we added a few indexers recently.
Is there something i should update in the search head when i add indexers?
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Unlocking Unified Insights: New Gigamon Federated Search App for Splunk
GA: New Data Management App in Splunk Platform
Announcing Modern Navigation: A New Era of Splunk User Experience
