One of my clients has an app in a container. I am unable to modify the index it goes to. I would like to put two of the sourcetypes that come from that host in a different index (by default it puts it in the main index). This is what I have so far, but wanted to know if this configuration would work:
props.conf
[mysourcetype1]
TRANSFORMS-index1 = overrideindex
[mysourcetype2]
TRANSFORMS-index1 = overrideindex
transforms.conf
[overrideindex1]
DEST_KEY = _MetaData:Index
REGEX = .
FORMAT = my_new_index
[overrideindex2]
DEST_KEY = _MetaData:Index
REGEX = .
FORMAT = my_other_new_index
For the application we created, the sourcetypes are going to the correct index. Unfortunately, the "default" container install on Diamonti, which we can't change, sends logs to the main index. So, because of that setup, I need to change on the indexers before the actual indexing is done. Since I can't control it with the app we push out to the containers, I need to have it happen once the indexers begin to receive the "default" sourcetypes from the containers.
Unfortunately I don't have access to the configuration on the diamonti containers so I can't put it in the inputs.conf. I have to catch these AFTER they are sent to the index (main). We have an inputs.conf on the container, but these are logs we couldn't control with inputs.conf, they are part of an automatic feed to the main index.
But how are you pushing these inputs.conf? I meant via deployment server or via any orchestration mechanism like ansible/puppet?
I think you have put the exact same name in both transforms stanza
[mysourcetype1]
TRANSFORMS-index1 = overrideindex1
[mysourcetype2]
TRANSFORMS-index2 = overrideindex2
Also, why are you trying to do at props/transforms level? This is more intensive right as every event needs to be transformed. It may be easy to assign a new sourcetype & different index at inputs.conf level itself in the client system
eg . in inputs.conf
[monitor:///var/log/file1.log]
index = my_new_index
sourcetype = mysourcetype1
[monitor:///var/log/file2.log]
index = my_other_new_index
sourcetype = mysourcetype2
Forgot to put the 1 and 2 in the props.conf above. So I have them as 1 & 2 props and 1 & 2 transforms