topic Re: What underlying database Splunk uese to operate? in All Apps and Add-ons

What underlying database Splunk uese to operate?

SomnathShilimka — Tue, 30 Apr 2013 13:40:10 GMT

Hi All,
i am newbie in splunk, i searched over documentation and also over community. but i did not get answer for question in my mind.
Question is :- What database Splunk uses to work ? like Oracle or MySQL or PostgreSQL ?

i also read one already posted question as provided on below link
http://splunk-base.splunk.com/answers/32499/what-database-engine-splunk-uses

on same link, in answers it is written like below :-
I am not sure what is unclear to you in the previous answer. Splunk uses it's own engine, and does not rely on any external databases in order to operate. It manages its own database via a series of flat files and indexes, and Damien has provided a few good resources for you to have an idea on how the data engine works.

does it means Splunk do not use any database like Oracle or MySQL etc ?

thanks & regards,
Somnath

Re: What underlying database Splunk uese to operate?

martin_mueller — Tue, 30 Apr 2013 13:44:04 GMT

Indeed, no separate relational DB. The data structure underneath is Splunk-built.

Re: What underlying database Splunk uese to operate?

SomnathShilimka — Tue, 30 Apr 2013 14:45:37 GMT

Hi Martin_mueller
thanks for reply, i know for splunk no need of separate DB, what i want to know is :- what is name of that Splunk-built database(data structure) ?

Regards,
Somnath

Re: What underlying database Splunk uese to operate?

martin_mueller — Tue, 30 Apr 2013 14:48:53 GMT

There is no separate product underneath, it's been built by Splunk.

Re: What underlying database Splunk uese to operate?

bspargur1 — Thu, 16 Feb 2017 14:24:59 GMT

There are the indexes which are flat files. The lookups are csv files or mongodb. The alerts require relationships and are also stored in a DB but I can't recall off hand if it is mysql or mongodb. I think it is mysql.

Re: What underlying database Splunk uese to operate?

nickhills — Thu, 16 Feb 2017 14:47:43 GMT

nope, mysql is not used inside splunk .
the kvstore is mongo, but again this is not used by splunk for anykind of relationship mapping or storage.\
The kvstore is provided for splunk applications to use not for splunk per se.

Re: What underlying database Splunk uese to operate?

stanwin — Thu, 16 Feb 2017 14:52:49 GMT

These links should clarify that:

http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/Howindexingworks

http://docs.splunk.com/Documentation/Splunk/5.0.1/Indexer/HowSplunkstoresindexes

Re: What underlying database Splunk uese to operate?

bspargur1 — Thu, 16 Feb 2017 14:55:29 GMT

You are looking specifically at the indexes. The kvstores are the mongodb. Do you have reference to how the Splunk generated alerts are stored?

Re: What underlying database Splunk uese to operate?

bspargur1 — Thu, 16 Feb 2017 15:01:32 GMT

I understand indexing but that is not the only place or way the Splunk stores data.

Re: What underlying database Splunk uese to operate?

bspargur1 — Thu, 16 Feb 2017 15:12:49 GMT

When you configure alerts in Splunk, where are they stored? How is the relationship to the event data that was used to generate the alert stored? 🙂

Splunk uses the mongodb to store data becasue csv files suck as they start to get large. I have always been curious why they used a json structured db over just using their flat file structure of the indexes with the mappings built in. Maybe because it is good and already existed?

Re: What underlying database Splunk uese to operate?

martin_mueller — Thu, 16 Feb 2017 16:10:04 GMT

Re: What underlying database Splunk uese to operate?

bspargur1 — Thu, 16 Feb 2017 16:17:21 GMT

Haha, that was an awesome pic. 🙂

Re: What underlying database Splunk uese to operate?

stanwin — Thu, 16 Feb 2017 18:38:08 GMT

Thats a biiiiig bucket 😄