Getting Data In

How to determine disk utilization of a source?

New Member

Hi,
Is there a way to determine how much disk space a source is using?

here is my index, source and sourcetype:

index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file 

Best Regards,
Chris

0 Karma
1 Solution

Champion

The is not real good way of doing this but you can make some assumptions. You can assume that every character in _raw is one byte (some unicode characters are four bytes). You then pick a time period you want to sample.

 index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file earliest=-8h@h | eval length=len(_raw) | stats avg(length) as bytesPerMessage

I don't know the size of your message but lets say every message on average 481 bytes. Now that you've done that you can use tstats find the count of those messages per day. Then we can use some eval statements with Math.

|tstats count WHERE index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file by _time span=1d | eval bytesPerDay=(481*count) | eval MB=(bytesPerDay/1024/1024)

This will get you close be is only an approximation.

View solution in original post

0 Karma

Champion

The is not real good way of doing this but you can make some assumptions. You can assume that every character in _raw is one byte (some unicode characters are four bytes). You then pick a time period you want to sample.

 index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file earliest=-8h@h | eval length=len(_raw) | stats avg(length) as bytesPerMessage

I don't know the size of your message but lets say every message on average 481 bytes. Now that you've done that you can use tstats find the count of those messages per day. Then we can use some eval statements with Math.

|tstats count WHERE index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file by _time span=1d | eval bytesPerDay=(481*count) | eval MB=(bytesPerDay/1024/1024)

This will get you close be is only an approximation.

View solution in original post

0 Karma

New Member

Thank you very much, bmacias84

0 Karma

Splunk Employee
Splunk Employee

@wasupchris - Did the answer provided by bmacias84 help provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

0 Karma

New Member

Anyone knows how to identify the average disk space utilization by data source.

0 Karma

Splunk Employee
Splunk Employee

@wasupchris - If you want to try to get some immediate help for your question, you should join the 1300+ Splunk users in our public Slack chat. People ask each other for immediate help on there daily. You can share your question/link to your post there to see if anyone can take a stab at it.

You first have to request access through www.splunk402.com/chat. Fill out the form, and once you receive the approval email from our Community Manager (usually the approval process takes a couple days), you can access Slack.com and ask for help in the #general channel or the #n00bs channel (if you're brand new to Splunk).

0 Karma