topic What are the execution sequence of transforms from different stanza located in the difference configuration files ? in Getting Data In

What are the execution sequence of transforms from different stanza located in the difference configuration files ?

fxyfrank_acn — Fri, 08 Feb 2019 02:47:26 GMT

We want to change sourcetype and then send data to two different Splunk Indexers.

What is happening is the sourcetype is getting changed (that means first transform is working) BUT the seconds pros.conf stanza present in the apps folder is not working (It is only send the logs to default output group).

Transform 1: SPLUNK_HOME/etc/system/local/
props.conf

[source::/abc/xyz.log]
TRANSFORMS-changesourcetype = st

transforms.conf

[st]
REGEX = \.*\[12345]\.*
FORMAT = sourcetype::my_sourcetype
DEST_KEY = MetaData:Sourcetype

Transform 2: SPLUNK_HOME/etc/apps/application/local/
props.conf

[my_sourcetype]
TRANSFORMS-routing = route_data

transforms.conf

[route_data]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = indexer1, indexer2

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

kamlesh_vaghela — Fri, 08 Feb 2019 06:13:18 GMT

@fxyfrank_acn

Please see How Splunk determines precedence order and other section for your answer.

https://docs.splunk.com/Documentation/Splunk/7.2.3/Admin/Wheretofindtheconfigurationfiles

You can run btool to see all the configuration values in use by your Splunk instance.

http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurations

Thanks

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

MuS — Fri, 08 Feb 2019 06:21:04 GMT

And there is the common misunderstanding:

btool does not show the actual config in use by Splunk, it merges all on disk config files and shows the potential configuration Splunk is using ....

Quote from the docs:

Btool displays merged on-disk configurations. That is, btool shows you the merged settings in the .conf files. It does not necessarily show you what Splunk software is currently using.

link to the docs https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurations

If you want to see the actual config Splunk is using right now, run this command:

$SPLUNK_HOME/bin/splunk show config ....

links to the docs https://docs.splunk.com/Documentation/Splunk/latest/Admin/CLIadmincommands

It is a bit like in the old days with Cisco routers, there is a difference between running config and start-up config 😉

cheers, MuS

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

vishaltaneja070 — Fri, 08 Feb 2019 07:20:22 GMT

Hello @fxyfrank_acn
Can you please share the details present in outputs.conf as well.

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

vishaltaneja070 — Fri, 08 Feb 2019 07:22:50 GMT

You have to mention something like this in outputs.conf as well to make second transforms work:

[tcpout:indexer1]
disabled=false
server=xx.x.xx.x:9997

[tcpout:indexer2]
disabled=false
server=xx.x.xx.x:9997

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

harsmarvania57 — Fri, 08 Feb 2019 09:08:22 GMT

Have a look at my answer https://answers.splunk.com/answers/686241/metadata-transforms-not-being-applied-after-series-1.html , you will get an idea what is happening.

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

fxyfrank_acn — Wed, 13 Feb 2019 03:43:40 GMT

the two indexers are specified in the outputs.conf as what you have mentioned however it still doesn't work.

I have tried to apply the Sourcetype change on the Indexer (indexing time), still no luck.

Re: What are the execution sequence of transforms from different stanza located in the difference configuration files ?

anwarmian — Tue, 27 Oct 2020 02:46:30 GMT

Since this will be performed at index parsing stage file precedence will be in global context so /system/local will have higher precedence than application/local. So, in your case "my_sourcetype" will be created first then you can use "my_sourcetype" in application/local to redirect logs to different indexes.