topic Re: _audit and _internal index data retention in Getting Data In

_audit and _internal index data retention

ualbanytech — Thu, 23 Jun 2011 22:59:36 GMT

EDIT: Splunk version = 4.1.6

Are there any guidelines on the length of time that _audit and _internal index data should be kept?

I have come up with age-out policies for our Splunk events, however
the part I'm stuck on is how long should I keep my _audit and _internal events?

My initial thought is to keep events in those two indexes for the same age as my oldest index (5 years).

The only problem is the majority of my indexes are only retained for 1 year or less.

Spacewise, it seems wasteful to keep all of _audit and _internal for 5 years.

Re: _audit and _internal index data retention

hexx — Fri, 24 Jun 2011 04:09:07 GMT

You certainly don't need to keep events from _internal and _audit for 6 years.

Events in _internal mostly are indexed from $SPLUNK_HOME/var/log/splunk. The majority of the volume comes from files such as splunkd.log and metrics.log.

The information contained in those events is typically interesting to troubleshoot Splunk-specific issues or to get sample measurements of event-processing thruput from metrics.log.

As it is rare to have to troubleshoot Splunk issues that are older than a month, I would say that the default retention period of 28 days set for _internal in $SPLUNK_HOME/etc/system/default/indexes.conf is adequate :

[_internal] homePath = $SPLUNK_DB/_internaldb/db coldPath = $SPLUNK_DB/_internaldb/colddb thawedPath = $SPLUNK_DB/_internaldb/thaweddb maxDataSize = 100 frozenTimePeriodInSecs = 2419200

The _audit index is where Splunk logs events from fschange inputs by default (see the File system change monitor section of inputs.conf.spec for more information - http://www.splunk.com/base/Documentation/latest/Admin/Inputsconf).

Events in this index are kept for 6 years by default (again, a setting inherited from $SPLUNK_HOME/etc/system/default/indexes.conf), but unless you have your own fschange inputs, only $SPLUNK_HOME/etc is audited in this way. For that reason, you could want to shorten the retention period for this index, although it is usually very small in size anyway.

Example : Let's modify $SPLUNK_HOME/etc/system/local/indexes.conf to set a retention period of 20 days for _internal and 60 days for _audit. We'll simply add the two following stanzas to that file :

[_internal] frozenTimePeriodInSecs = 1728000

[_audit] frozenTimePeriodInSecs = 5184000

Re: _audit and _internal index data retention

ualbanytech — Fri, 24 Jun 2011 15:28:28 GMT

Follow up question. I created my local policies by placing indexes.conf in

SPLUNK_HOME/etc/system/local

I just verified the default/indexes.conf has the policies as you outlined for _internal and _audit

However, my _internal and _audit indexes do not appear to be obeying the policies.

I did not re-create stanzas in my local indexes.conf as it was my understanding that
any I define in local overrides those in default dir.

Manager >> Indexes shows
_audit 3,548 MB w/ earliest Dec 30, 2009
_internal 5,193 MB w/ earliest Dec 4, 2009

Re: _audit and _internal index data retention

hexx — Fri, 24 Jun 2011 16:51:05 GMT

You still need to declare the stanza for which you are changing the parameters from the default. I have amended my answer above to provide a clear example of what should go into the local version of indexes.conf.

Re: _audit and _internal index data retention

ualbanytech — Fri, 24 Jun 2011 20:10:50 GMT

I wasn't clear enough. I did not change the defaults for _internal yet the default
frozenTimePeriodInSecs has been exceeded by over a year.

It seems my problem relates to how Splunk ages out data (only when rolling between buckets).
And, that is contingent on other settings.

The age out based on time is too complicated.

Based on your answer, I decided to just set the max size to 1 GB and 2 GB for _audit and _internal (respectively).

Thank You very much!

Re: _audit and _internal index data retention

splunksriniwipr — Fri, 11 Sep 2015 09:44:12 GMT

Hi,
splunk maintains its default settings in $SPLUNK_HOME/etc/system/default path...

If you want to make any changes on default properties, then you can create inputs.conf or index.conf etc conf files under /etc/system/local/ direcotry....

use same stanza's in *.conf files. with different values... Hope It will helpful

Thanks,
Srinivas

Re: _audit and _internal index data retention

buckiboy — Wed, 02 Aug 2017 12:02:38 GMT

Guys, are we sure its called _audit. Looking at our indexers, the directory is called audit.

Re: _audit and _internal index data retention

x3mboy — Tue, 28 Jan 2020 21:04:27 GMT

I had changed the $SPLUNK_HOME/etc/system/local/indexes.conf to change _internal and _audit size, but when I try the bundle-push, it fails saying "No new bundle will be pushed. The master and peers already have this bundle with bundle id = xxxxx"

How changes in $SPLUNK_HOME/etc/system/local/ should be pushed to the indexers?

Re: _audit and _internal index data retention

rtev — Thu, 14 May 2020 18:47:22 GMT

Make the changes in $SPLUNK_HOME/etc/master-apps/local/indexes.conf on the Master and Splunk should recognize it needs a new bundle.

Re: _audit and _internal index data retention

tomasmoser — Sat, 28 Nov 2020 19:53:00 GMT

Yes, it's correct. See below.

splunk@test:/opt/splunk/var/lib/splunk$ splunk btool indexes list _audit | grep audit [_audit] coldPath = $SPLUNK_DB/audit/colddb homePath = $SPLUNK_DB/audit/db thawedPath = $SPLUNK_DB/audit/thaweddb tstatsHomePath = volume:_splunk_summaries/audit/datamodel_summary

I agree it's not logical and Splunk should change directory name from "audit" to "_audit" on a filesystem.

Re: _audit and _internal index data retention

splunkreal — Mon, 08 Mar 2021 13:41:13 GMT

Solved this with maxTotalDataSizeMB

Re: _audit and _internal index data retention

SamHTexas — Tue, 27 Apr 2021 14:47:13 GMT

Thank u for your post. Am asked for a document to prove that Splunk Audit logs are kept for 1 year. Where do I find such a document & edit it if necessary? Thank u in advance.