Splunk Dev

Sourcetype reassignment with props and transforms not working

chrismmckenna
New Member

I'm trying to create a very basic sourcetype override. The inputs.conf on a set of forwarders have been set with a wildcard directory stanza so the sourcetype is the same for all sources.

List of sources and sourcetypes:

source  sourcetype
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV1.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-101.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-102.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.iadx-app-103.sec.bfnet.us.log   checkout-api
/opt/logs/checkout-api-proxy/thirdparty.log checkout-api

On the indexer/search head
$SPLUNK_HOME/etc/system/local/props.conf: (all contents listed, no other outputs.conf with same stanza)

[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
TRANSFORMS-checkout-api-v2 = checkout-api-iadx-v2

$SPLUNK_HOME/etc/system/local/transforms.conf: (all contents listed, no other transforms.conf with same stanza)
WRITE_META = True

[checkout-api-iadx-v2]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::checkout-v2
REGEX = .*

New events don't have a modified sourcetype. What am I missing here?

0 Karma

sloshburch
Splunk Employee
Splunk Employee

The index override needs to be where the data gets cooked. That will be on the Indexer or on the Heavy Forwarder (if one such HF exists before the Indexers). The Search Head will apply search-time override but use of the META keys means you're trying to do this at the point where the data gets cooked.

Also, I'm not sure you need SOURCE_KEY if you are following the documented approach in http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

0 Karma

woodcock
Esteemed Legend

You are MASSIVELY over-complicating this. Just do this in your props.conf and put it on your indexers:

[source::/opt/logs/checkout-api-proxy/checkoutV2.iadx-app-101.sec.bfnet.us.log]
sourcetype = checkout-v2

chrismmckenna
New Member

I've tried the following as well: http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Bypassautomaticsourcetypeassignment

props.conf (on indexer/search head)

[source::.../checkoutV2.*]
sourcetype = checkout-v2

Is there something in default/props.conf that is preventing this from working as expected?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Did you update the outputs.conf OR props.conf?? It should be props.conf BTW.

0 Karma

chrismmckenna
New Member

I updated props.conf and transforms.conf, creating a new version of each in $SPLUNK_HOME/etc/system/local.

I used this link as a guideline. https://answers.splunk.com/answers/942/source-typing-and-transforms.html

Should I not use SOURCE_KEY in the transform? Or, is my REGEX off?

0 Karma

chrismmckenna
New Member
0 Karma

ssadanala1
Contributor
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...