Splunk Search
Highlighted

how to classify events by source when they have the same sourcetpye?

Communicator

Hi

I'm indexing a file which is being written by a syslog process (therefore I defined the sourcetype=syslog) and this has events from different sources(origins). However the events are very similar and I want to tell Splunk how to "tag" this events by source at index time. My problem is not separate the events to different indexes but rather just classify them as different sources using a regex.

I have in inputs.conf:

[monitor:///data/syslog_cache]
index = my_index
sourcetype = syslog

I 'm thinking that I could do this in props.conf and transforms.conf but it doesn't work.

props.conf

[syslog]
EXTRACT-mySource1 = regex1
TRANSFORMS-set = myset1    
EXTRACT-mySource2 = regex2
TRANSFORMS-set = myset2

transforms.conf

[myset1]
source = mysource1

[myset2]
source = mysouce2

I'm probably confusing the goal behind this stanzas and attributes and I would appreciate any help.

0 Karma
Highlighted

Re: how to classify events by source when they have the same sourcetpye?

Splunk Employee
Splunk Employee

Hi, it seems that you have the pieces, but in the wrong order.

props.conf


[syslog]
TRANSFORMS-set = myset1,myset2, myset3
# the transforms will apply in the order, and the last one can possibly replace the modifications from the previous ones.

transforms.conf


[myset1]
SOURCEKEY = _raw
# optional, this is the default
REGEX = <your
regex>
DESTKEY = MetaData:Source
FORMAT = source::$1
# beware the caps are important, no caps for "source" in format, initial cap for "Source" and "Metadata" in Dest
key

[myset2]
# etc...

View solution in original post

Highlighted

Re: how to classify events by source when they have the same sourcetpye?

Communicator

What's the specific role of "FORMAT = source::$1" ? Don't quite understand this part.

0 Karma
Highlighted

Re: how to classify events by source when they have the same sourcetpye?

Splunk Employee
Splunk Employee

It's going to set the source based on the regex you provide.

0 Karma
Highlighted

Re: how to classify events by source when they have the same sourcetpye?

Communicator

I wasn't sure about "::$1" but nevermind.

Good example. Thank you for your assistance.

0 Karma
Highlighted

Re: how to classify events by source when they have the same sourcetpye?

Splunk Employee
Splunk Employee

Without specifications, the FORMAT goes directly to the _raw field.

But if you are using it to modify a metadata field (host, source, index, etc....) then you have to use the format
source::mysource (for a static source)
or in the case or a dynamic regex match

REGEX=blahblah(mymatch)blahblah
FORMAT=myfield::$1
the $1 $2, etc will be the different matches in the Regex.