Knowledge Management
Highlighted

collect index="based on values"

New Member

Hi everybody,

is it possible to create several summary index within one search?

Example:
"Index A" has a field "OS" with values "Windows", "Linux"...

Is there a way to tell splunk to loop something like: index=A | collect index="OS" (where OS is the field-value)

So that each result based on field OS will be stored in (allready created) index "Windows", "Linux", ....

Many thanks in advance.

0 Karma
Highlighted

Re: collect index="based on values"

Legend

Hi fklink,
why do you need to use only one search?
Bye.
Giuseppe

0 Karma
Highlighted

Re: collect index="based on values"

New Member

Hi Guiseppe,

the values Windows and Linux are exemplary.

In fact, there are about 150 different values that vary.

My goal is to use one query to fill the 150 different indexes (automatically)

0 Karma
Highlighted

Re: collect index="based on values"

Legend

Hi fklink,
are you sure about this?
an elevated quantity of indexes isn't a best practice!
You should analyze your needs and define the correct number of indexes thinking to the main reason to have different indexes:

  • different retention periods,
  • different access rights.

In addition you can take in consideration also the quantity of logs ingested (e.g. it isn't a good practice to have in the same index logs from large flows with logs from little flows).
But anyway 150 indexes are very many indexes!
Don't think to indexes as DB tables, Splunk is different!

Bye.
Giuseppe

0 Karma
Highlighted

Re: collect index="based on values"

New Member

Hi Guiseppe,

thanks for your help.

Because of permissions i have to create 150 index 😞

Maybe i will find an other solution.....

0 Karma
Highlighted

Re: collect index="based on values"

SplunkTrust
SplunkTrust

Here's how I've written that before

search that generates events with some field rectype

| rename COMMENT as "send foo to index foo"
| appendpipe [ 
    | where rectype="foo" 
    | collect index="foo" testmode=f other parameters
    | where false()
    ]

| rename COMMENT as "send bar to index bar"
 | appendpipe [ 
    | where rectype="bar" 
    | collect index="bar" testmode=f other parameters
    | where false()
    ]

| rename COMMENT as "repeat for each destination index"


| rename COMMENT as "This last one doesn't go to an index but it lets you see as the search results what would have done so for all types"
| collect index="goingnowhere" testmode=t other parameters 
0 Karma
Highlighted

Re: collect index="based on values"

New Member

Hi DalJeanis,

many thanks for your help.

Since the values within "rectype" can differ, I'm looking for a solution that fills the field variably.

someting like: | collect index="$value_of_rectype$"

0 Karma
Highlighted

Re: collect index="based on values"

Esteemed Legend

Like this:

search that generates events with some field rectype
| _SI_Name_{index} = "IgnoreMe"
| foreach _SI_Name_* [
| appendpipe [ 
   where rectype="$MATCHSTR$"
| collect index="$MATCHSTR$" testmode=f other parameters
| where false()
] ]
| rename COMMENT as "Anything left at this point did not go into an index, which may be an error"
0 Karma