Deployment Architecture

how is data stored in Splunk?

MaryCampbell
New Member

I hope you don't mind but I have a couple of more questions which have come back to me:

Volume, one of our customers has 1 million CDRs per day would there be any issue with this in Splunk?

We zip the older files with bzip, can these be read in the same way as non zipped files

How is data stored in Splunk is in a database?

In a distributed setting how would remote files be transmitted SFTP? Are these compressed first.

Thanking you in advance Mary

Tags (1)
0 Karma

Ayn
Legend

No, volume is not an issue. Splunk has been deployed in environments where it's handling several terabytes a day. It all comes down to sizing your Splunk deployment correctly. More info here: http://docs.splunk.com/Documentation/Splunk/5.0.1/installation/capacityplanningforalargersplunkdeplo...

bzip2-compressed files can be read in the same way as non-compressed ones.

Data is stored in a Splunk-specific kind of database known as an "index". More information available (among other places) here: http://docs.splunk.com/Documentation/Splunk/5.0.1/Indexer/HowSplunkstoresindexes

In a distributed environment you usually install Splunk forwarders (agents) on the systems data is to be read from. Communication between the forwarders and the indexer is done using a proprietary protocol that can be encrypted and compressed. More information available here: http://docs.splunk.com/Documentation/Splunk/5.0.1/Deploy/Aboutforwardingandreceivingdata

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...