Deployment Architecture

Monitoring csv file using Universal forwarder missing some data into index

c_krishna_gutur
Explorer

I have Task Scheduler which runs for every 6 hours and generates .csv file into a folder, I am monitoring this file using universal forwarder, for the last 15 to 20 days it works as expected, now the indexing is not happening completely i.e it is not taking all the records into index Ex: My .csv file contains 800 records but it is indexing around 225 records, why is this behaviour ? Any thoughts ?

Tags (1)
0 Karma

hunters_splunk
Splunk Employee
Splunk Employee

Hi Krishna,

Please try the following:

  1. Stop your indexer and forwarder.
  2. On the indexer, reset the csv input checkpoint, use the btprobe command: splunk cmd btprobe –d SPLUNK_HOME/var/lib/splunk/ fishbucket/splunk_private_db --file --reset
  3. Start your indexer and forwarder.

Hope it helps. Thanks!
Hunter

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

From Data to Insight: Announcing the Winners of the Splunk Dashboard Contest

Hi Splunkers, First off, thank you to everyone who participated in our very first From Data to Insight: The ...

Splunk Developers: Construct Your Future at the .conf26 Builder Bar

Calling all Splunk architects, platform admins, and app developers: the site is open, and the blueprints are ...

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...