My data flows in from the forwarder where index=idx1 and sourcetype=sourcetypeA have been set using inputs.conf.
On the indexer, I want the following rule to be set for the same data, (i.e. I want to change the sourcetype based on the value of source)
How do I set up this rule on the indexer?
I am using Splunk Enterprise.
Provide an initial sourcetype for your data in your inputs.conf. This will remain the sourcetype if none of your transforms stanzas match. In your props.conf you point at the different stanza names in your transforms.conf file (setsourcetype1 and setsourcetype2 in my example).
In your props.conf:
[mysourcetype] TRANSFORMS-set_sourcetype = set_sourcetype_1, set_sourcetype_2
In your transforms.conf:
[set_sourcetype_1] SOURCE_KEY = MetaData:Source REGEX = ^source::/path/to/the/file/source1.txt FORMAT = sourcetype::sourcetype_1 DEST_KEY = MetaData:Sourcetype [set_sourcetype_2] SOURCE_KEY = MetaData:Source REGEX = ^source::/path/to/the/file/source2.txt FORMAT = sourcetype::sourcetype_2 DEST_KEY = MetaData:Sourcetype
In this example, you are routing it to the appropriate transforms.conf stanzas based on the original sourcetype. Each transforms.conf stanza looks at the source field and determines if the REGEX matches. If it does, it sets the sourcetype to the value after the :: in the FORMAT = sourcetype:: line.
Thanks for the response.
Unfortunately my forwarders are owned by a different team, and they are not willing to change anything for now. I will try the above settings and see if it works.
Ok. If that is the case, then you could use the props.conf and transforms.conf method I describe, however, it would probably be more efficient doing it at the Universal Forwarder side if you can get them to make the change in the future.
why not on the forwarder?
[monitor:///abc/logs/server-*-error.log] index = idx1 sourcetype=sourcetypeBError [monitor:///abc/logs/server-*-transaction.log] index = idx1 sourcetype=sourcetypeBTransaction [monitor:///abc/logs/server-*-access.log] index = idx1 sourcetype=sourcetypeBAccess
if you insist setting it up on the indexer
there are many answers here on this subject, here are couple examples:
hope it helps