Splunk Search

how to classify events by source when they have the same sourcetpye?

fernandoandre
Communicator

Hi

I'm indexing a file which is being written by a syslog process (therefore I defined the sourcetype=syslog) and this has events from different sources(origins). However the events are very similar and I want to tell Splunk how to "tag" this events by source at index time. My problem is not separate the events to different indexes but rather just classify them as different sources using a regex.

I have in inputs.conf:

[monitor:///data/syslog_cache]
index = my_index
sourcetype = syslog

I 'm thinking that I could do this in props.conf and transforms.conf but it doesn't work.

props.conf

[syslog]
EXTRACT-mySource1 = regex1
TRANSFORMS-set = myset1    
EXTRACT-mySource2 = regex2
TRANSFORMS-set = myset2

transforms.conf

[myset1]
source = mysource1

[myset2]
source = mysouce2

I'm probably confusing the goal behind this stanzas and attributes and I would appreciate any help.

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Hi, it seems that you have the pieces, but in the wrong order.

props.conf


[syslog]
TRANSFORMS-set = myset1,myset2, myset3
# the transforms will apply in the order, and the last one can possibly replace the modifications from the previous ones.

transforms.conf


[myset1]
SOURCE_KEY = _raw
# optional, this is the default
REGEX =
DEST_KEY = MetaData:Source
FORMAT = source::$1
# beware the caps are important, no caps for "source" in format, initial cap for "Source" and "Metadata" in Dest_key

[myset2]
# etc...

View solution in original post

yannK
Splunk Employee
Splunk Employee

Hi, it seems that you have the pieces, but in the wrong order.

props.conf


[syslog]
TRANSFORMS-set = myset1,myset2, myset3
# the transforms will apply in the order, and the last one can possibly replace the modifications from the previous ones.

transforms.conf


[myset1]
SOURCE_KEY = _raw
# optional, this is the default
REGEX =
DEST_KEY = MetaData:Source
FORMAT = source::$1
# beware the caps are important, no caps for "source" in format, initial cap for "Source" and "Metadata" in Dest_key

[myset2]
# etc...

yannK
Splunk Employee
Splunk Employee

Without specifications, the FORMAT goes directly to the _raw field.

But if you are using it to modify a metadata field (host, source, index, etc....) then you have to use the format
source::mysource (for a static source)
or in the case or a dynamic regex match

REGEX=blahblah(mymatch)blahblah
FORMAT=myfield::$1
the $1 $2, etc will be the different matches in the Regex.

fernandoandre
Communicator

I wasn't sure about "::$1" but nevermind.

Good example. Thank you for your assistance.

0 Karma

sdaniels
Splunk Employee
Splunk Employee

It's going to set the source based on the regex you provide.

0 Karma

fernandoandre
Communicator

What's the specific role of "FORMAT = source::$1" ? Don't quite understand this part.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...