Archive
Highlighted

Sourcetype reassignment with props and transforms not working

New Member

I'm trying to create a very basic sourcetype override. The inputs.conf on a set of forwarders have been set with a wildcard directory stanza so the sourcetype is the same for all sources.

List of sources and sourcetypes:

source  sourcetype
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.log checkout-api

On the indexer/search head
$SPLUNK_HOME/etc/system/local/props.conf: (all contents listed, no other outputs.conf with same stanza)

[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
TRANSFORMS-checkout-api-v2 = checkout-api-iadx-v2

$SPLUNK_HOME/etc/system/local/transforms.conf: (all contents listed, no other transforms.conf with same stanza)
WRITE_META = True

[checkout-api-iadx-v2]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::checkout-v2
REGEX = .*

New events don't have a modified sourcetype. What am I missing here?

0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

Contributor
0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

SplunkTrust
SplunkTrust

Did you update the outputs.conf OR props.conf?? It should be props.conf BTW.

0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

New Member

I updated props.conf and transforms.conf, creating a new version of each in $SPLUNK_HOME/etc/system/local.

I used this link as a guideline. https://answers.splunk.com/answers/942/source-typing-and-transforms.html

Should I not use SOURCE_KEY in the transform? Or, is my REGEX off?

0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

New Member
0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

New Member

I've tried the following as well: http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Bypassautomaticsourcetypeassignment

props.conf (on indexer/search head)

[source::.../checkoutV2.*]
sourcetype = checkout-v2

Is there something in default/props.conf that is preventing this from working as expected?

0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

Esteemed Legend

You are MASSIVELY over-complicating this. Just do this in your props.conf and put it on your indexers:

[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
sourcetype = checkout-v2
0 Karma
Highlighted

Re: Sourcetype reassignment with props and transforms not working

Ultra Champion

The index override needs to be where the data gets cooked. That will be on the Indexer or on the Heavy Forwarder (if one such HF exists before the Indexers). The Search Head will apply search-time override but use of the META keys means you're trying to do this at the point where the data gets cooked.

Also, I'm not sure you need SOURCE_KEY if you are following the documented approach in http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.