topic Re: How to determine disk utilization of a source? in Getting Data In

How to determine disk utilization of a source?

wasupchris — Mon, 06 Mar 2017 21:10:05 GMT

Hi,
Is there a way to determine how much disk space a source is using?

here is my index, source and sourcetype:

index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file

Best Regards,
Chris

Re: How to determine disk utilization of a source?

wasupchris — Mon, 06 Mar 2017 22:54:06 GMT

Anyone knows how to identify the average disk space utilization by data source.

Re: How to determine disk utilization of a source?

aaraneta_splunk — Mon, 06 Mar 2017 22:58:37 GMT

@wasupchris - If you want to try to get some immediate help for your question, you should join the 1300+ Splunk users in our public Slack chat. People ask each other for immediate help on there daily. You can share your question/link to your post there to see if anyone can take a stab at it.

You first have to request access through www.splunk402.com/chat. Fill out the form, and once you receive the approval email from our Community Manager (usually the approval process takes a couple days), you can access Slack.com and ask for help in the #general channel or the #n00bs channel (if you're brand new to Splunk).

Re: How to determine disk utilization of a source?

bmacias84 — Mon, 06 Mar 2017 23:37:14 GMT

The is not real good way of doing this but you can make some assumptions. You can assume that every character in _raw is one byte (some unicode characters are four bytes). You then pick a time period you want to sample.

 index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file earliest=-8h@h | eval length=len(_raw) | stats avg(length) as bytesPerMessage

I don't know the size of your message but lets say every message on average 481 bytes. Now that you've done that you can use tstats find the count of those messages per day. Then we can use some eval statements with Math.

|tstats count WHERE index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file by _time span=1d | eval bytesPerDay=(481*count) | eval MB=(bytesPerDay/1024/1024)

This will get you close be is only an approximation.

Re: How to determine disk utilization of a source?

wasupchris — Tue, 07 Mar 2017 00:04:38 GMT

Thank you very much, bmacias84

Re: How to determine disk utilization of a source?

aaraneta_splunk — Thu, 09 Mar 2017 23:07:40 GMT

@wasupchris - Did the answer provided by bmacias84 help provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!