Getting Data In

Tranformations to set different sourcetypes based on fields in an event

pksudip
New Member

I am trying to change the sourcetype on the events from a dataset based on certain fields in the dataset that is currently being added using a scripted input. This is what I have currently:

props.conf


[source::testservice]
TRANSFORMS-changesourcetype = sourcetype-test1info, sourcetype-test2info

transforms.conf


[sourcetype-test1info]
DEST_KEY = MetaData:SourceType
REGEX = "field1=(?[^ ])"
FORMAT = sourcetype::test1info
CLEAN_KEYS = 0
MV_ADD = 0


[sourcetype-test2info]
DEST_KEY = MetaData:SourceType
REGEX = "field2=(?[^ ]
)"
FORMAT = sourcetype::test2info
CLEAN_KEYS = 0
MV_ADD = 0

The files are currently located in etc/apps/appname/local. I dont see this transformation having any effect. The event would look something like:
2011-06-01 20:41:13 PDT timestamp=1306986073 field1=value1 location=testlocation

Any idea what I may be missing?

Tags (1)
0 Karma
1 Solution

cgilbert_splunk
Splunk Employee
Splunk Employee

I believe that your DEST_KEY value isn't valid....keys are case-sensitive and MetaData:Sourcetype is the correct value, not MetaData:SourceType

From: $SPLUNK_HOME/etc/system/README/transforms.conf.spec

*******

KEYS:

*******

  • NOTE: Keys are case-sensitive. Use the following keys exactly as they appear.

queue : Specify which queue to send the event to (can be parsingQueue, nullQueue, indexQueue).
_raw : The raw text of the event.
_done : If set to any string, this represents the last event in a stream.
_meta : A space-separated list of metadata for an event.
_time : The timestamp of the event, in seconds since 1/1/1970 UTC.
MetaData:FinalType : The event type of the event.

MetaData:Host : The host associated with the event.
The value must be prefixed by "host::"

_MetaData:Index : The index where the event should be stored.

MetaData:Source : The source associated with the event.
The value must be prefixed by "source::"

MetaData:Sourcetype : The sourcetype of the event.
The value must be prefixed by "sourcetype::"

View solution in original post

cgilbert_splunk
Splunk Employee
Splunk Employee

I believe that your DEST_KEY value isn't valid....keys are case-sensitive and MetaData:Sourcetype is the correct value, not MetaData:SourceType

From: $SPLUNK_HOME/etc/system/README/transforms.conf.spec

*******

KEYS:

*******

  • NOTE: Keys are case-sensitive. Use the following keys exactly as they appear.

queue : Specify which queue to send the event to (can be parsingQueue, nullQueue, indexQueue).
_raw : The raw text of the event.
_done : If set to any string, this represents the last event in a stream.
_meta : A space-separated list of metadata for an event.
_time : The timestamp of the event, in seconds since 1/1/1970 UTC.
MetaData:FinalType : The event type of the event.

MetaData:Host : The host associated with the event.
The value must be prefixed by "host::"

_MetaData:Index : The index where the event should be stored.

MetaData:Source : The source associated with the event.
The value must be prefixed by "source::"

MetaData:Sourcetype : The sourcetype of the event.
The value must be prefixed by "sourcetype::"

pksudip
New Member

That was the issue, after fixing that I also realized that the regex shouldn't be in double quotes. Thanks for the quick response!!

0 Karma
Get Updates on the Splunk Community!

Developer Spotlight with Paul Stout

Welcome to our very first developer spotlight release series where we'll feature some awesome Splunk ...

State of Splunk Careers 2024: Maximizing Career Outcomes and the Continued Value of ...

For the past four years, Splunk has partnered with Enterprise Strategy Group to conduct a survey that gauges ...

Data-Driven Success: Splunk & Financial Services

Splunk streamlines the process of extracting insights from large volumes of data. In this fast-paced world, ...