Deployment Architecture

how is data stored in Splunk?

MaryCampbell
New Member

I hope you don't mind but I have a couple of more questions which have come back to me:

Volume, one of our customers has 1 million CDRs per day would there be any issue with this in Splunk?

We zip the older files with bzip, can these be read in the same way as non zipped files

How is data stored in Splunk is in a database?

In a distributed setting how would remote files be transmitted SFTP? Are these compressed first.

Thanking you in advance Mary

Tags (1)
0 Karma

Ayn
Legend

No, volume is not an issue. Splunk has been deployed in environments where it's handling several terabytes a day. It all comes down to sizing your Splunk deployment correctly. More info here: http://docs.splunk.com/Documentation/Splunk/5.0.1/installation/capacityplanningforalargersplunkdeplo...

bzip2-compressed files can be read in the same way as non-compressed ones.

Data is stored in a Splunk-specific kind of database known as an "index". More information available (among other places) here: http://docs.splunk.com/Documentation/Splunk/5.0.1/Indexer/HowSplunkstoresindexes

In a distributed environment you usually install Splunk forwarders (agents) on the systems data is to be read from. Communication between the forwarders and the indexer is done using a proprietary protocol that can be encrypted and compressed. More information available here: http://docs.splunk.com/Documentation/Splunk/5.0.1/Deploy/Aboutforwardingandreceivingdata

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...