topic Re: Storage question in Getting Data In

Storage question

attgjh1 — Wed, 23 May 2012 01:20:30 GMT

Where does the indexed data gets stored?
Do they somehow get reduplicated when the logs are indexed?

i.e. I'm planning to use splunk to monitor up to 5gig of data daily. All the logs are sent to a server which a forwarder will forward to PC to index the data and run searches.

From the scenario above, is it necessary to set it up this way? or i can do everything via web splunk from any PC while the Indexer can be directly located with the server?

Re: Storage question

sdaniels — Wed, 23 May 2012 01:40:11 GMT

The indexes will get stored on your indexer server. In your case a combined Indexer/Search Head. Forwarders will send the data from the servers to the indexer and it will be stored there. See the docs for details.

http://docs.splunk.com/Documentation/Splunk/latest/admin/HowSplunkstoresindexes

Architecture: http://docs.splunk.com/Documentation/Splunk/latest/installation/Splunksarchitectureandwhatgetsinstalled

Re: Storage question

attgjh1 — Wed, 23 May 2012 01:43:35 GMT

thanks for the fast reply. 😃

so technically, if im testing data on my PC now, for every 5 gig of data i index from the harddisk, another 5gig or so worth of indexed data will also be created and stored on the indexer, which is taking up my harddisk space?

just confused with this part as i need to factor in server space required to consider acquiring splunk 🙂

Re: Storage question

sdaniels — Wed, 23 May 2012 01:52:30 GMT

Typically it is compressed at 50% - 60% of the raw data but depends on the data and if you are using Summary indexing. So you'll store around 2.5+ GB per day based on the raw data of 5 GB per day.