Getting Data In

How to create a new Sourcetype from Event Data not the raw data?

robertlynch2020
Motivator

Hi

I am getting in the below data (green box in image).

In green is the raw data and in purple is the event data. 

The issue is there are 3 source types in one and I need a way to separate them into 3 source types using transforms (Or something like that). 

However as the data is event data, how do I do that?

For example, in the past when I had to create a new source type  I could use something like this.

 

 

[AMBER_RAW]
SEDCMD-remove_header = s/^.*?\{/{/1
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
TRANSFORMS-sourcetye_routing = AMBER_RAW_json_EVENT,AMBER_RAW_json_TRACE,AMBER_RAW_json_METRIC
EXTRACT-CLUSTER_MACHINE_TEST = ^(?:[^\[\n]*\[){2}(?P<CLUSTER_MACHINE_TEST>[^/]+)

 

 

 

robertlynch2020_0-1652778993534.png

This shows the three different source types that is possible. So i need to create 3 different ones form the orginal one.

robertlynch2020_1-1652779260800.png

 

 

Labels (2)
Tags (2)
0 Karma

PickleRick
Ultra Champion

I'm not entirely sure what you mean since you already know how to overwrite sourcetype but you can use SOURCE_KEY option in your transform to match your regex against a specific field. One caveat though - it must be a indexed field. You cannot use search-time extractions for it.

0 Karma

gcusello
Legend

Hi @robertlynch2020,

you can override sourcetype value finding a regex to identify each one.

You can do this on Heavy Forwarders (if present) on the Indexers, following the instructions at https://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides#:~:text=You%20c....

In few words,

you have to create a stanza in props.conf

[<your_original_sourcetype>]
TRANSFORMS-<class> = <unique_stanza_name>

and a stanza in transforms.conf

[<unique_stanza_name>]
REGEX = <your_regex>
FORMAT = sourcetype::<your_custom_sourcetype_value>
DEST_KEY = MetaData:Sourcetype

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...