Hello @dural_yyz, here are some updates ! So the above approach didn't work as expected... The thing is after I changed the sourcetype of my events, they will directly go to the indexing phase, I do not think it's possible to "tell them to go back to parsing queue" in order to apply a CLONE_SOURCETYPE afterwards... So I tried something else to clone and forward (to another Splunk HF) only a subset of my events... (nasty but for now I have no other options...). The main idea is to change the _TCP_ROUTING value for the event I want to clone, those events will be selected with the REGEX and directly forward to the wanted destination. Afterwards I will clean the events that were not changed by my REGEX (with an "inverse" REGEX) by sending them to null queue. Here is an example with a JSON incoming raw data (I assigned them the sourcetype "mysourcetype") : {
"Foo": "Bar",
"Hello": "World"
},
{
"Foo": "Bar",
"Hello": "Again"
} props.conf [mysourcetype]
TRANSFORMS-foo-clone = trans-clone
[mysourcetype:cloned]
TRANSFORMS-bar-drop = trans-drop transforms.conf # clone all events and change tcp output for specific events mathcing the regex
[trans-clone]
CLONE_SOURCETYPE = mysourcetype:cloned
REGEX = "Hello":\s*"World"
DEST_KEY = _TCP_ROUTING
FORMAT = my_specific_output
# drop duplicated and not forwarded logs
[trans-drop]
REGEX = (?s)^(?!.*"Workload"\s*:\s*"Aip").*$
DEST_KEY = queue
FORMAT = nullQueue I tested it and do the job for me ! I do not like the fact to use two "opposite REGEX" because of resources usage and if your REGEX is not okay, you will end up with duplicated unwanted data. Thanks you for your time and @isoutamo for the good hint about dropping some data 😋 GaetanVP
... View more