How to Forward different subsets to different indexes?

[rvk_sp_user]

Hello Splunk Community,

I am pretty new to Splunk and I have a use case where different subsets of logs are to be forwarded to different indexes from the same monitor location. For example all logs matching a pattern(like a regex) should go to index1 and all other logs should go to default index.

I found this Splunk documentation but this is forwarding logs from different monitor locations.

I have the following in inputs.conf..

inputs.conf
[monitor:///<monitor-location>]
index = <my-index>
sourcetype = <type>

Can you please help?

[rvk_sp_user]

@gcusello Thanks for the answer. But, I already have a working solution of forwarding logs with stanzas to an index. What I am essentially looking for is how to send a subset of the logs from the same input to a different index; while sending all other logs to the default index. 

[gcusello]

Hi @rvk_sp_user,

in this case, you need a regex to identify the logs to send to an index different than the default then you have to override index value based on that regex.

in few words, on your Indexers or (if present) or your Heavy Forwarders, not on Universal Forwarders, you have to put in

props.conf

#props.conf 
[mysourcetype]
TRANSFORMS-index = overrideindex

transforms.conf

# transforms.conf 
[overrideindex]
DEST_KEY =_MetaData:Index
REGEX = <your_regex>
FORMAT = your_new_index

Ciao.

Giuseppe

[rvk_sp_user]

Sorry for late response. Thanks a lot @gcusello . My requirement changed a bit.

So, here 

DEST_KEY =_MetaData:Index

for "Index" needs to be replaced with my old index?

[gcusello]

Hi @rvk_sp_user,

No, when an event with the sourcetype you defined in props.conf (mysourcetype) arrives to the Indexer or (if present) to the Heavy Forwarder, it's checked if the event contains the regex that you defined in transforms.conf:

• if the regex matches, the value for index that you fixed in inputs.conf is overwritten with the new value that you defined in transforms.conf (your_new_index),
• if it doesn't matches, original index value remains.

This means that in inputs.conf you have to put the default value for index and in transforms.conf you have to put the new index value associated to a regex.

If you have to assign more index values, you have to create more stanzas in props.conf and transforms.conf with different regexes.

Ciao.

Giuseppe

[gcusello]

Hi @rvk_sp_user,

in each stanza of your inputs.conf, you can add an option to say which is the index that has to receive data from that input.

in each stanza you have to insert 

[monitor://var/log]
index=index_for_that_input
sourcetype=sourcetype_for_that_input

you could follow the instruction about getting data in at https://www.google.com/search?q=splunk+getting+data+in&rlz=1C1SQJL_itIT832IT832&oq=splunk+getting+da...

There are videos and documentation.

Ciao.

Giuseppe