How to check total logs size in MB being sent by a host

[splunktp]

Hey Guys,

Our Netflow monitoring system shows that most of the bandwidth is being consumed by port 9997 coming from a remote site with Splunk Forwarder and Head Office with Splunk Indexer.

What is the correct and accurate search query to for getting the total log size (in MB) being sent by a host to the Splunk Indexer?

In Splunkweb, what is the search query to drill down the top log sources (in MB) in a 1 week period?

Thanks,
Mark

[joshd]

Definitely support using the Deployment Monitor ... I've also written a short blog post where I extracted a lot of the queries so you can individually schedule or run them and alert as needed... see it here:

http://www.joshd.ca/content/splunk-usage-statistic-searches

[MuS]

hi splunktp

here are two examples how you could get how much was transfered.
if your universal forwarder and forwarder also forward internal logs, you can list the total size of all internal logs transfered for each log file with this command

index=_internal source=*metrics* | stats sum(kb) by series 

the second command lists the raw size of all event and sums them up for each log file

| eval raw_len=(len(_raw)/1028) | stats sum(raw_len) by source 

hope this helps

cheers

[MuS]

oh sorry missed the MB in your question, the examples return everything in KB 😉

[BrianKJr]

you do /1028 in your example isn't it /1024 ? or am I missing something that Splunk does with padding?

[kristian_kolb]

I think you should look into the SplunkDeploymentMonitor, as it comes with a bunch of predefined searches that let you look up these things.

http://docs.splunk.com/Documentation/Splunk/4.3/Deploy/Startthedeploymentmonitor

If you for some reason do not wish to do that, you can still get some information from Status -> Index Activity -> Indexing volume or more directly http(s)://your_server/en-GB/app/search/indexing_volume

There you can see the indexing broken down over host, source, sourcetype etc for an arbitrary period of time.

Hope this helps,

Kristian

[proletariat99]

I downvoted this post because pretty tired of hearing "use the deployment monitor". in a distributed, scaled environment, not everyone has access to that app. This glib response is equivalent to Splunk's standard "RTFM" answer.