Sourcetype reassignment with props and transforms not working

[chrismmckenna]

I'm trying to create a very basic sourcetype override. The inputs.conf on a set of forwarders have been set with a wildcard directory stanza so the sourcetype is the same for all sources.

List of sources and sourcetypes:

source  sourcetype
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.log checkout-api

On the indexer/search head
$SPLUNK_HOME/etc/system/local/props.conf: (all contents listed, no other outputs.conf with same stanza)

[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
TRANSFORMS-checkout-api-v2 = checkout-api-iadx-v2

$SPLUNK_HOME/etc/system/local/transforms.conf: (all contents listed, no other transforms.conf with same stanza)
WRITE_META = True

[checkout-api-iadx-v2]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::checkout-v2
REGEX = .*

New events don't have a modified sourcetype. What am I missing here?

[sloshburch]

The index override needs to be where the data gets cooked. That will be on the Indexer or on the Heavy Forwarder (if one such HF exists before the Indexers). The Search Head will apply search-time override but use of the META keys means you're trying to do this at the point where the data gets cooked.

Also, I'm not sure you need SOURCE_KEY if you are following the documented approach in http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

[woodcock]

You are MASSIVELY over-complicating this. Just do this in your props.conf and put it on your indexers:

[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
sourcetype = checkout-v2

[chrismmckenna]

I've tried the following as well: http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Bypassautomaticsourcetypeassignment

props.conf (on indexer/search head)

[source::.../checkoutV2.*]
sourcetype = checkout-v2

Is there something in default/props.conf that is preventing this from working as expected?

[somesoni2]

Did you update the outputs.conf OR props.conf?? It should be props.conf BTW.

[chrismmckenna]

I updated props.conf and transforms.conf, creating a new version of each in $SPLUNK_HOME/etc/system/local.

I used this link as a guideline. https://answers.splunk.com/answers/942/source-typing-and-transforms.html

Should I not use SOURCE_KEY in the transform? Or, is my REGEX off?

[chrismmckenna]

I've also followed this documentation: http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Advancedsourcetypeoverrides

[ssadanala1]

Please have a look at this question

https://answers.splunk.com/answers/36932/sourcetytping-and-override-source-name-on-directory-with-mu...