- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why am I seeing a lot of name=cooked_output events in _internal?
All of a sudden, noticed getting tons of events in _internal with name=cooked_output. What could be causing this behavior?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This event is logged when Splunk sends data over the network. Data can be sent in two ways 1) cooked-when it is sent to another splunk instance and 2)uncooked-when it is being sent somewhere else like to a syslog server. My guess is this the log of a universal forwarder which is sending data to a Splunk indexer. The number of these events will scale proportional to the universal forwarders. They are benign and not a cause for concern. They are provided for informational reasons
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your answer, Craig. The thing that was troubling is that starting on June 18, we have gone from about 10 of these per day to around 1-2 million cooked_output events per day.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Interesting. Is there anything that happened on the day that changed in your infrastructure i.e Splunk upgrades, new hosts, major config changes?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
No. We've talked to the infrastructure guys and the last patches were before the behavior started by a month or so. So the number of cooked seemed excessively high comparatively.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hmm. If there are no Warn or Error messages I don't think that it is anything benign for now. It could be caused by changes in the logging behavior/frequency of a particular log that is being monitored. In my opinion, keep an eye on your environment for more WARN or ERROR messages that would be a clearer indicator that something is wrong.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could you perhaps post the entire event line with source and sourcetype information?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yes...
This is typically how it looks:
07-19-2016 15:18:53.994 -0500 INFO Metrics - group=thruput, name=cooked_output, instantaneous_kbps=0.300961, instantaneous_eps=0.354828, average_kbps=0.397792, total_k_processed=489797.000000, kb=9.330078, ev=11.000000