Splunk Search

How to check total logs size in MB being sent by a host

splunktp
Explorer

Hey Guys,

Our Netflow monitoring system shows that most of the bandwidth is being consumed by port 9997 coming from a remote site with Splunk Forwarder and Head Office with Splunk Indexer.

What is the correct and accurate search query to for getting the total log size (in MB) being sent by a host to the Splunk Indexer?

In Splunkweb, what is the search query to drill down the top log sources (in MB) in a 1 week period?

Thanks,
Mark

Tags (1)
0 Karma

joshd
Builder

Definitely support using the Deployment Monitor ... I've also written a short blog post where I extracted a lot of the queries so you can individually schedule or run them and alert as needed... see it here:

http://www.joshd.ca/content/splunk-usage-statistic-searches

0 Karma

MuS
SplunkTrust
SplunkTrust

hi splunktp

here are two examples how you could get how much was transfered.
if your universal forwarder and forwarder also forward internal logs, you can list the total size of all internal logs transfered for each log file with this command

index=_internal source=*metrics* | stats sum(kb) by series 

the second command lists the raw size of all event and sums them up for each log file

| eval raw_len=(len(_raw)/1028) | stats sum(raw_len) by source 

hope this helps

cheers

MuS
SplunkTrust
SplunkTrust

oh sorry missed the MB in your question, the examples return everything in KB 😉

0 Karma

BrianKJr
Explorer

you do /1028 in your example isn't it /1024 ? or am I missing something that Splunk does with padding?

0 Karma

kristian_kolb
Ultra Champion

I think you should look into the SplunkDeploymentMonitor, as it comes with a bunch of predefined searches that let you look up these things.

http://docs.splunk.com/Documentation/Splunk/4.3/Deploy/Startthedeploymentmonitor

If you for some reason do not wish to do that, you can still get some information from Status -> Index Activity -> Indexing volume or more directly http(s)://your_server/en-GB/app/search/indexing_volume

There you can see the indexing broken down over host, source, sourcetype etc for an arbitrary period of time.

Hope this helps,

Kristian

proletariat99
Communicator

I downvoted this post because pretty tired of hearing "use the deployment monitor". in a distributed, scaled environment, not everyone has access to that app. This glib response is equivalent to Splunk's standard "RTFM" answer.

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>