Monitoring Splunk

How does indexing affect hard drive space?

sthao
New Member

I am indexing to the main index and it has a max size of 500000 MB defined. So far, I have indexed about 14,000 MB of data. What I noticed is that 14 GB of my hard drive was taken up right away and it appears to correlate to the 14,000 MB of data indexed. At this time, I only have 40 GB total on my hard drive. So, does this mean that the 500000 MB max size defined won't apply because I only have 40000 MB of hard drive space? Would I need to increase my hard drive to 500 GB to fully utilize the 500000 MB definition?

Thanks for any insight/direction that can be given!

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

This is within the expected size range:

http://www.splunk.com/base/Documentation/latest/Installation/HowHowmuchspaceyouwillneed

Typical ASCII syslog data takes about 50%, but it could be anywhere from 10% to 200%, with a typical range from 20% to 120% of the original source data size.

0 Karma

sthao
New Member

I was not able to get the "for" line executions to go, but I did setup a temporary Splunk install elsewhere to analyze the data storage size of one of my data inputs. It turned out that the reported data stored was about 3 times the size of the actual data read in.

So, I am correct in assuming that I would definitely need 500 GB to fully utilize the default 500000 MB max size set for indexes right?

0 Karma

sthao
New Member

Thanks for the response. I am trying to run the Windows command line section and nothing is appearing after each of the "for" line executions. Would you have any suggestions? Thanks again!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...