Getting Data In

Where is forwarded data stored in the indexer after getting indexed?

Explorer

Hi Team,

Where are the forwarded logs being saved in the indexer after getting indexed?
As i know this is very known issue but still i did not get my answer for it.

in general, indexes.conf contain below details :-

Cold    $SPLUNK_HOME/var/lib/splunk/defaultdb/colddb/*
Hot             $SPLUNK_HOME/var/lib/splunk/defaultdb/db/*
Thawed  $SPLUNK_HOME/var/lib/splunk/defaultdb/thaweddb/*  

But as per my indexes.conf file i can able to see :-

coldPath = $SPLUNK_DB/audit/colddb
homePath = $SPLUNK_DB/audit/db
thawedPath = $SPLUNK_DB/audit/thaweddb

so here is the confusion in the path, it should be $SPLUNK_HOME or $SPLUNK_DB ?

if it is $SPLUNK_HOME then please find the below details cause $SPLUNK_HOME= /opt/product/splunk :-

bash-3.2$ pwd
/opt/product/splunk/var/lib/splunk
bash-3.2$ ls -lrt
total 44
drwx------ 5 XYZ XYZ 4096 Jul  2  2012 summarydb
drwx------ 5 XYZ XYZ 4096 Jul  2  2012 _internaldb
drwx------ 5 XYZ XYZ 4096 Jul  2  2012 historydb
drwx------ 2 XYZ XYZ 4096 Jul  2  2012 hashDb
drwx------ 5 XYZ XYZ 4096 Jul  2  2012 defaultdb
drwx------ 5 XYZ XYZ 4096 Jul  2  2012 blockSignature
drwx------ 2 XYZ XYZ 4096 Jul  2  2012 authDb
drwx------ 5 XYZ XYZ 4096 Jul  2  2012 audit
drwx--x--- 4 XYZ XYZ 4096 Jul  2  2012 appserver
drwx------ 2 XYZ XYZ 4096 Jul  2  2012 persistentstorage
drwx------ 7 XYZ XYZ 4096 Jul  3  2012 fishbucket

and i am not able to see the forwarded logs over here.
or if it is $SPLUNK_DB then where can i see the full path of it?

Thanks,
Seema

0 Karma

SplunkTrust
SplunkTrust

Hi seema2502,

check your $SPLUNK_HOME/etc/splunk-launch.conf for the $SPLUNK_DB setting.
If unset, becomes $SPLUNK_HOME/var/lib/splunk (unix) or %SPLUNK_HOME%\var\lib\splunk (windows)

cheers, MuS

Explorer

Hi Mus,

Thanks for the quick response.
yes i am able to see my $SPLUNK_DB path inside $SPLUNK_HOME/etc/splunk-launch.conf.

when i checked inside the path found the below details:-

/apps/splunk/data/var/lib/splunk
bash-3.2$ du -sh *
3.1G audit
4.0K authDb
20K blockSignature
416G defaultdb
27M fishbucket
4.0K hashDb
20K historydb
2.4G _internaldb
1.2M persistentstorage
20K repolite_idx
20K summarydb
29M summary_forwarders
39M summary_hosts
15M summary_indexers
17M summary_pools
116M summary_sources
29M summary_sourcetypes

As defaultdb is having 416G size i went inside the defaultdb directory

/apps/splunk/data/var/lib/splunk/defaultdb
bash-3.2$ du -sh *
4.0K colddb
416G db
4.0K thaweddb

As db is having 416G size i went inside the db directory
/apps/splunk/data/var/lib/splunk/defaultdb/db

can you please confirm, are these logs the same which are being indexed after getting forwarded from forwarder.

Thanks,
Seema

0 Karma

SplunkTrust
SplunkTrust

each directory within /apps/splunk/data/var/lib/splunk represents an index, each file within /apps/splunk/data/var/lib/splunk/defaultdb/db represents a bucket (your events or data) of your index=main

see the docs for more details http://docs.splunk.com/Documentation/Splunk/6.1.4/Indexer/HowSplunkstoresindexes

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!