I'm trying to create a very basic sourcetype override. The inputs.conf on a set of forwarders have been set with a wildcard directory stanza so the sourcetype is the same for all sources.
List of sources and sourcetypes:
source sourcetype
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-101.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-102.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-103.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-102.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-103.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-101.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-102.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-103.sec.bfnet.us.log checkout-api
/opt/logs/checkout-api-proxy/thirdparty.log checkout-api
On the indexer/search head
$SPLUNK_HOME/etc/system/local/props.conf
: (all contents listed, no other outputs.conf with same stanza)
[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
TRANSFORMS-checkout-api-v2 = checkout-api-iadx-v2
$SPLUNK_HOME/etc/system/local/transforms.conf
: (all contents listed, no other transforms.conf with same stanza)
WRITE_META = True
[checkout-api-iadx-v2]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::checkout-v2
REGEX = .*
New events don't have a modified sourcetype. What am I missing here?
The index override needs to be where the data gets cooked. That will be on the Indexer or on the Heavy Forwarder (if one such HF exists before the Indexers). The Search Head will apply search-time override but use of the META keys means you're trying to do this at the point where the data gets cooked.
Also, I'm not sure you need SOURCE_KEY
if you are following the documented approach in http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides
You are MASSIVELY over-complicating this. Just do this in your props.conf and put it on your indexers:
[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
sourcetype = checkout-v2
I've tried the following as well: http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Bypassautomaticsourcetypeassignment
props.conf
(on indexer/search head)
[source::.../checkoutV2.*]
sourcetype = checkout-v2
Is there something in default/props.conf that is preventing this from working as expected?
Did you update the outputs.conf OR props.conf?? It should be props.conf BTW.
I updated props.conf and transforms.conf, creating a new version of each in $SPLUNK_HOME/etc/system/local
.
I used this link as a guideline. https://answers.splunk.com/answers/942/source-typing-and-transforms.html
Should I not use SOURCE_KEY
in the transform? Or, is my REGEX off?
I've also followed this documentation: http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Advancedsourcetypeoverrides
Please have a look at this question