I happen to have some control over our java logs. Rather than use transforms.conf/props.conf to create various sourcetypes based on the app that is running. Could I simply log out sourcetype in my logs? I believe Splunk will respect it as a KVP. But I don't know if it will really treat it like a sourcetype?
Splunk uses sourcetypes to know how to parse logs. Without a sourcetype, how will Splunk know how to parse your log to find the sourcetype? Why needlessly index and store another KVP in your events? I'm not saying it can't be done, just that you're probably not saving yourself anything.
I agree with @richgalloway that you gain little by doing this. If you want to take control of your logs, it would be better to implement a naming convention for the log files and directories. Then you could easily assign the sourcetype at input time.
In particular, if you group log files of the same sourcetype into the same directory, you can make entries in the
inputs.conf like this example:
Assigning the sourcetype in
inputs.conf is the most efficient way to set the sourcetype. Another alternative (about as efficient) is to include the sourcetype name in the file name. For example: if you had an Apache log named xyz, you could instead name it
Then you could make the following entry in
props.conf for each sourcetype
You can certainly add something to your log files. You could even use
transforms.conf to create a transform that assigns the sourcetype to each event based on that value - but this is very inefficient! It will significantly slow the indexing of the data as the sourcetype will have to be extracted and assigned to each event, rather than to the entire file at once.
Actually you can do this, by adding this code into your logs:
Then the forwarder will set the sourcetype dinamically, but as Richard said, you still need (or should) define the way the indexer parses that sourcetype, so you will need to have defined in advance all possible sourcetypes you would want to define dinamically in the log itself.