Monitoring Splunk

How does indexing affect hard drive space?

sthao
New Member

I am indexing to the main index and it has a max size of 500000 MB defined. So far, I have indexed about 14,000 MB of data. What I noticed is that 14 GB of my hard drive was taken up right away and it appears to correlate to the 14,000 MB of data indexed. At this time, I only have 40 GB total on my hard drive. So, does this mean that the 500000 MB max size defined won't apply because I only have 40000 MB of hard drive space? Would I need to increase my hard drive to 500 GB to fully utilize the 500000 MB definition?

Thanks for any insight/direction that can be given!

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

This is within the expected size range:

http://www.splunk.com/base/Documentation/latest/Installation/HowHowmuchspaceyouwillneed

Typical ASCII syslog data takes about 50%, but it could be anywhere from 10% to 200%, with a typical range from 20% to 120% of the original source data size.

0 Karma

sthao
New Member

I was not able to get the "for" line executions to go, but I did setup a temporary Splunk install elsewhere to analyze the data storage size of one of my data inputs. It turned out that the reported data stored was about 3 times the size of the actual data read in.

So, I am correct in assuming that I would definitely need 500 GB to fully utilize the default 500000 MB max size set for indexes right?

0 Karma

sthao
New Member

Thanks for the response. I am trying to run the Windows command line section and nothing is appearing after each of the "for" line executions. Would you have any suggestions? Thanks again!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Mastering Threat Intelligence in ES 8.5, Splunk AI Assistant v2, and More from Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

Break the Build: Inside the KubeDoom Lounge at .conf26

    You step up to the machine. The pixelated corridors of a certain 1993 FPS load in front of you, EMP Pulse ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...