Hello Splunkers1!
I am encountering an issue with field extraction related to the sourcetype. My requirement is to map all values from the EVENTTYPEKEY field into the sourcetype.
At search time, using the SPL query| eval sourcetype=EVENTTYPEKEY
works correctly, and the values align as expected.
However, I want to implement this at the index-time level. To achieve this, I have configured props.conf and transforms.conf, but the values are not being populated in the sourcetype field.
Could you please advise where the configuration might be incorrect or what needs to be adjusted to make this work at index time?
============================================
Below are settings
transforms.conf:
[rewrite_sourcetype_wmc]
REGEX= EVENTTYPEKEY="(\w+)"
REGEX= TYPE="(\w+)"
FORMAT=sourcetype::$1
DEST_KEY=MetaData:Sourcetype
props.conf :
[wmc_events]
CHARSET=AUTO
KV_MODE=AUTO
SHOULD_LINEMERGE=false
category=Vanderlande
description= WMC events received from the Oracle database, formatted as key-value pairs
pulldown_type=true
TIME_PREFIX = ^TIMESTAMP=
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6NZ
TZ = UTC
NO_BINARY_CHECK = true
TRUNCATE = 10000000
#MAX_EVENTS = 100000
ANNOTATE_PUNCT = false
TRANSFORMS-assignSourcetype = rewrite_sourcetype_wmc
[source::tcp:8097]
KV_MODE = AUTO
LINE_BREAKER = (\003)
[wmc_log]
REPORT-wmc_logs = REPORT-wmc_logs
SHOULD_LINEMERGE = true
MUST_BREAK_AFTER = (\003)
CHARSET = AUTO
NO_BINARY_CHECK = false
OK. I see some questionable things here.
1. You can't use two REGEX entries in one transform stanza. One will overwrite the other (check with btool your effective configuration).
2. I see you have different SHOULD_LINEMERGE settings for different wmc_* sourcetype. I assume at least one of those is supposed to be that "recast" one. It won't work this way. Line breaking and merging occurs way before any transforms are fired. See the Masa diagrams https://community.splunk.com/t5/Getting-Data-In/Diagrams-of-how-indexing-works-in-the-Splunk-platfor...
3. Have you assigned the "basic" wmc_events sourcetype on the input level.?
As already said, you cannot use two REGEX in one stanza.
If you have this kind of need you probably need do use INGEST_EVAL where you can use several SPL eval and based on those set then sourcetype=xyz from previous phases.
Can you give some example events to us?
Hi @isoutamo Below are some sample logs. I want all the values under the field EVENTTYPEKEY to come to sourcetype as well.
TIMESTAMP="2026-04-02T08:04:18.076000Z",SOURCE="Login.loginOperator",IDEVENT="74098806",EVENTTYPEKEY="OPERATOR_LOGIN",INSTANCEID="56040561",OPERATORID="1",OPERATORLOGIN="1"
TIMESTAMP="2026-04-02T08:04:13.588000Z",SOURCE="TSUManager.createTsu",IDEVENT="74098786",EVENTTYPEKEY="TSU_CREATED",FULLYQUALIFIEDNAME="RCV/RcvInduct/EntryA4",INSTANCEID="130011301",LOCATIONID="10000000922",LOCATIONTEXT="RCV/RcvInduct/EntryA4",OPERATOR="1",OPERATORID="1",TASKID="",TSUID="10660306773",TSULABEL="111000000001938900",TSUROLE="SINGLESKU",TSUTYPE="10000281844",TSUTYPELABEL="CHEP"
TIMESTAMP="2026-04-02T08:04:13.636000Z",SOURCE="MfPalletInduct_x.handleMessage",IDEVENT="74098789",EVENTTYPEKEY="PALLET_INDUCT",CONTAINERLABEL="",INSTANCEID="130011301",OPERATOR="1",OPERATORID="1",SCANMODE="AUTO",TSULABEL="111000000001938900",TSUROLE="PRODUCT",WORKSTATION_LOCATION="RCV/RcvInduct/EntryA4"
EVENTTYPEKEY="OPERATOR_LOGIN"
EVENTTYPEKEY="TSU_CREATED"
EVENTTYPEKEY="PALLET_INDUCT"
This should work when you update your transforms.conf to
[rewrite_sourcetype_wmc]
REGEX= EVENTTYPEKEY="([^\"]+)"
FORMAT=sourcetype::$1
DEST_KEY=MetaData:SourcetypeBe sure that this is in 1st full splunk instance from source!
In props.conf you should have
[wmc_events]
...
TIME_PREFIX = ^TIMESTAMP="
...
TRANSFORMS-assignSourcetype = rewrite_sourcetype_wmcAre you sure that you are needing CHARSET and TZ here?
Those are usually needed on source node where your inputs.conf is.
Also NO_BINARY_CHECK should be in source system.
Hi @uagraw01 ,
how do you assign the "wmc_events" sourcetype (that you're using in your props.conf) to your data?
try to use [source::tcp:8097]
Ciao.
Giuseppe
Hi @gcusello , Do you mean I need to use below setting in props.conf ?
[source::tcp:8097]
TRANSFORMS-assignSourcetype = rewrite_sourcetype_wmc
KV_MODE = AUTO
LINE_BREAKER = (\003)
Hi @uagraw01 ,
yes, try it: in your props.conf the TRANSFORMS-assignSourcetype = rewrite_sourcetype_wmc is associates with the "wmc_events" sourcetype that it isn't associated with your data.
Ciao.
Giuseppe
Hi @uagraw01 ,
where did you located the props and transforms?
thy must be located on the first Full Splunk Instance (HF ot IDX, not UF) your data pass through, otherwise it doesn't run.
Ciao.
Giuseppe
Hi @gcusello , I am using a standalone server that acts as an indexer and search head both and props & transforms are both located on the same server.