https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243852#M72576 <P>You may also get what you need from something like this:<BR /> | metadata type=hosts index=<YOURINDEXNAME></YOURINDEXNAME></P> <P>It will return a totalCount column per host. If your forwarders are the original source of your log events, the event count should accurately reflect what's coming from each forwarder.</P> Mon, 28 Sep 2015 17:15:45 GMT s2_splunk 2015-09-28T17:15:45Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243849#M72573 <P>How can I determine which forwarder is impacting the indexer the most?<BR /> I have an index taking up 53 gigs of space with an event count of 296 million.<BR /> There are multiple forwarders feeding into this index.<BR /> The forwarders with the most events have directories that are less than 2 gigs in size.<BR /> I am manually going server to server to try and determine 'what is using all the space?'</P> Mon, 28 Sep 2015 15:14:14 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243849#M72573 MikeBertelsen 2015-09-28T15:14:14Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243850#M72574 <P>Hi, the _internal index should have this information. You can try something like:<BR /> <CODE>index=_internal metrics "group=tcpin_connections"</CODE></P> <P>"sourceHost" would be the forwarder... you extract per your requirement. Eg<BR /> <CODE>index=_internal earliest=-15m metrics "group=tcpin_connections"|stats sum(tcp_Kprocessed) by sourceHost</CODE><BR /> or tcp_eps (check out the <A href="http://docs.splunk.com/Documentation/Splunk/6.2.0/Troubleshooting/Aboutmetricslog" target="_blank">docs</A> for additional options)</P> Tue, 29 Sep 2020 07:22:58 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243850#M72574 Yasaswy 2020-09-29T07:22:58Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243851#M72575 <P>If you are using 6.2 or higher you can use DMC (Distributed Management Console). Here is the raw search you may want</P> <PRE><CODE>index=_internal host=lyn-del-spl-101 source="*metrics.log" sourcetype=splunkd group=per_host_thruput | timechart per_second(kb) as per_second sum(kb) as kb by series useother=false limit=15 </CODE></PRE> Mon, 28 Sep 2015 17:12:30 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243851#M72575 bmacias84 2015-09-28T17:12:30Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243852#M72576 <P>You may also get what you need from something like this:<BR /> | metadata type=hosts index=<YOURINDEXNAME></YOURINDEXNAME></P> <P>It will return a totalCount column per host. If your forwarders are the original source of your log events, the event count should accurately reflect what's coming from each forwarder.</P> Mon, 28 Sep 2015 17:15:45 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243852#M72576 s2_splunk 2015-09-28T17:15:45Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243853#M72577 <P>To clarify I'm interested in the data versus the events. Can a filepath containing 2 gigs spawn 10 gigs of disk space on an index?</P> <P>In this case 12 hosts are involved. Each apparently has files that total about 2 gigs. So at best I would expect that the index size on disk would be about 16 gigs on a daily basis. </P> <P>Putting it another way, SoS reports that this index is consuming on average 30 gigs of data per day. Where is the other 14 gigs coming from?</P> Mon, 28 Sep 2015 19:27:15 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243853#M72577 MikeBertelsen 2015-09-28T19:27:15Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243854#M72578 <P>Mike,<BR /> it is technically possible, but seems a bit out of the ordinary. Raw data usually compresses very nicely on disk, we frequently see compression rates beyond 75%.<BR /> The remaining disk space is used for the indexes and metadata that go along with the raw data. If you, for example, configured INDEXED_EXTRACTIONS = json/xml/etc. and you have very high cardinality in your source data, the size of the index files can quickly exceed the raw data size on disk.</P> <P>In other words: We need a bit more details on how you have your inputs configured for these 12 hosts.<BR /> You can also look at the directory structure on the indexer to see if you have multiple large .tsidx files.</P> <P>Hope this helps.<BR /> Stefan </P> Mon, 28 Sep 2015 19:35:17 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243854#M72578 s2_splunk 2015-09-28T19:35:17Z https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243855#M72579 <P>Turns out the individual that set up the monitoring on these twelve servers didn't exclude log rolling.<BR /> I'm going to clean up the monitoring and if the issue persists i will seek assistance another day.<BR /> Thanks for the answers and support.</P> <P>Mike</P> Tue, 29 Sep 2015 21:22:40 GMT https://community.splunk.com/t5/Splunk-Search/How-to-search-which-forwarder-is-sending-the-most-data-to-an/m-p/243855#M72579 MikeBertelsen 2015-09-29T21:22:40Z