Getting Data In

Tranformations to set different sourcetypes based on fields in an event

pksudip
New Member

I am trying to change the sourcetype on the events from a dataset based on certain fields in the dataset that is currently being added using a scripted input. This is what I have currently:

props.conf


[source::testservice]
TRANSFORMS-changesourcetype = sourcetype-test1info, sourcetype-test2info

transforms.conf


[sourcetype-test1info]
DEST_KEY = MetaData:SourceType
REGEX = "field1=(?[^ ])"
FORMAT = sourcetype::test1info
CLEAN_KEYS = 0
MV_ADD = 0


[sourcetype-test2info]
DEST_KEY = MetaData:SourceType
REGEX = "field2=(?[^ ]
)"
FORMAT = sourcetype::test2info
CLEAN_KEYS = 0
MV_ADD = 0

The files are currently located in etc/apps/appname/local. I dont see this transformation having any effect. The event would look something like:
2011-06-01 20:41:13 PDT timestamp=1306986073 field1=value1 location=testlocation

Any idea what I may be missing?

Tags (1)
0 Karma
1 Solution

cgilbert_splunk
Splunk Employee
Splunk Employee

I believe that your DEST_KEY value isn't valid....keys are case-sensitive and MetaData:Sourcetype is the correct value, not MetaData:SourceType

From: $SPLUNK_HOME/etc/system/README/transforms.conf.spec

*******

KEYS:

*******

  • NOTE: Keys are case-sensitive. Use the following keys exactly as they appear.

queue : Specify which queue to send the event to (can be parsingQueue, nullQueue, indexQueue).
_raw : The raw text of the event.
_done : If set to any string, this represents the last event in a stream.
_meta : A space-separated list of metadata for an event.
_time : The timestamp of the event, in seconds since 1/1/1970 UTC.
MetaData:FinalType : The event type of the event.

MetaData:Host : The host associated with the event.
The value must be prefixed by "host::"

_MetaData:Index : The index where the event should be stored.

MetaData:Source : The source associated with the event.
The value must be prefixed by "source::"

MetaData:Sourcetype : The sourcetype of the event.
The value must be prefixed by "sourcetype::"

View solution in original post

cgilbert_splunk
Splunk Employee
Splunk Employee

I believe that your DEST_KEY value isn't valid....keys are case-sensitive and MetaData:Sourcetype is the correct value, not MetaData:SourceType

From: $SPLUNK_HOME/etc/system/README/transforms.conf.spec

*******

KEYS:

*******

  • NOTE: Keys are case-sensitive. Use the following keys exactly as they appear.

queue : Specify which queue to send the event to (can be parsingQueue, nullQueue, indexQueue).
_raw : The raw text of the event.
_done : If set to any string, this represents the last event in a stream.
_meta : A space-separated list of metadata for an event.
_time : The timestamp of the event, in seconds since 1/1/1970 UTC.
MetaData:FinalType : The event type of the event.

MetaData:Host : The host associated with the event.
The value must be prefixed by "host::"

_MetaData:Index : The index where the event should be stored.

MetaData:Source : The source associated with the event.
The value must be prefixed by "source::"

MetaData:Sourcetype : The sourcetype of the event.
The value must be prefixed by "sourcetype::"

pksudip
New Member

That was the issue, after fixing that I also realized that the regex shouldn't be in double quotes. Thanks for the quick response!!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...