Splunk Search

Is there any disadvantage to keeping certain events in separate indexes?

nk-1
Path Finder

Everything goes into the default "main" index now.
I'm thinking of moving IIS log events into a new index called "iis" and Windows WMI events into a new index called "wmi".

Other than having to add the "index=" clause to my searches, dashboards, etc. is there any disadvantage to keeping these events in separate indexes?
(The docs indicate the advantages: control user access, accommodate varying retention policies, speed searches in certain situations)

1 Solution

dwaddle
SplunkTrust
SplunkTrust

Generally the answer here is "no up to a point". Separate indexes usually make life easier, not harder. If you configure things correctly, you don't even need to specify "index=" - simply define the indexes as "default searched" and it should just work.

Additional indexes are great until you go overboard with it. Each index searched can add some delay to a search. So, if you have 723 indexes you've probably went too far.

Also each index increases the number of open files that splunkd must keep. There is a finite limit here, so you'll need to know when to adjust it.

The number of indexes is pretty much a "Golidlocks" setting - too many is bad, too few is bad. You want to be "just right".

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

Generally the answer here is "no up to a point". Separate indexes usually make life easier, not harder. If you configure things correctly, you don't even need to specify "index=" - simply define the indexes as "default searched" and it should just work.

Additional indexes are great until you go overboard with it. Each index searched can add some delay to a search. So, if you have 723 indexes you've probably went too far.

Also each index increases the number of open files that splunkd must keep. There is a finite limit here, so you'll need to know when to adjust it.

The number of indexes is pretty much a "Golidlocks" setting - too many is bad, too few is bad. You want to be "just right".

nk-1
Path Finder

Thanks for the tip about adding the indexes as "default searched" !

bill_bartlett
Path Finder

The only disadvantage is knowing which indexes you have available. This is easily solvable by adding the indexes to the default search for the appropriate roles, but that's the only downside that I can think of. You've listed some of the advantages already, but it's also important to note that separating indexes allows for greater ease in the scaling of indexing volume across your cluster, if you ever end up needing it.

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...