Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Why did my heavy forwarder stopped forwarding data...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why did my heavy forwarder stopped forwarding data abruptly?
The HF in our environment stops forwarding data abruptly, while the Splunk process is still running. Initial investigation shows no issues with HF. On Restarting the HF it enables the data forwarding again before abruptly stops again with no clues in splunkd.log. There is no specific time no network issues.
Anyone with similar issues? any solutions?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How exactly do you come to the conclusion that your HF has stopped forwarding data?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Like anyone else, data from the particular host is more than 24hrs late. After restart, all that missing data appears and updated to latest time.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Im pretty sure this is not exactly the solution but try turning on the "Store a local copy" for lets say 5 minutes to verify that data is actually injecting into your heavy forwarder. then also login to the splunk console and then try telnetting into the receiving indexer's receiver port to make sure its not a network issue. Normally if there is some errors, there used to be a popup complaining that the receiver downstream is not receiving data and the data flow is stopped or something down the line like that. If both of the things are working fine then it might be something else.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The Web page for forwarder is disabled. hence no means to get that popup. I'll try your suggestion of store local copy
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Are you sure it stopped forwarding data?
If it is processing large files, it can could be running in "batchmode". What means it will try to send out the complete file before switching to the next.
And how many parallelIngestionPiplines are you runnig? It the system in in batchmode, nothing else will come though. When adding an extra pipleline, the "extra pipeline" can process the other data (will need more CPU prower)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
parallelIngestionPiplines =3
I understand batch mode, but then a couple of times I have waited for 1-2 days for data to arrive and data only starts arriving after restart
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Announcing Modern Navigation: A New Era of Splunk User Experience
Modernize your Splunk Apps – Introducing Python 3.13 in Splunk
Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...
