topic Re: how to classify events by source when they have the same sourcetpye? in Splunk Search

how to classify events by source when they have the same sourcetpye?

fernandoandre — Thu, 26 Apr 2012 13:29:37 GMT

I'm indexing a file which is being written by a syslog process (therefore I defined the sourcetype=syslog) and this has events from different sources(origins). However the events are very similar and I want to tell Splunk how to "tag" this events by source at index time. My problem is not separate the events to different indexes but rather just classify them as different sources using a regex.

I have in inputs.conf:

[monitor:///data/syslog_cache]
index = my_index
sourcetype = syslog

I 'm thinking that I could do this in props.conf and transforms.conf but it doesn't work.

props.conf

[syslog]
EXTRACT-mySource1 = regex1
TRANSFORMS-set = myset1    
EXTRACT-mySource2 = regex2
TRANSFORMS-set = myset2

transforms.conf

[myset1]
source = mysource1

[myset2]
source = mysouce2

I'm probably confusing the goal behind this stanzas and attributes and I would appreciate any help.

Re: how to classify events by source when they have the same sourcetpye?

yannK — Mon, 28 Sep 2020 11:44:32 GMT

Hi, it seems that you have the pieces, but in the wrong order.

props.conf



[syslog]

TRANSFORMS-set = myset1,myset2, myset3

 # the transforms will apply in the order, and the last one can possibly replace the modifications from the previous ones.

transforms.conf



[myset1]

SOURCE_KEY = _raw

 # optional, this is the default

REGEX = 

DEST_KEY = MetaData:Source

FORMAT = source::$1

 # beware the caps are important, no caps for "source" in format, initial cap for "Source" and "Metadata" in Dest_key

[myset2]
# etc...

Re: how to classify events by source when they have the same sourcetpye?

fernandoandre — Fri, 27 Apr 2012 13:40:26 GMT

What's the specific role of "FORMAT = source::$1" ? Don't quite understand this part.

Re: how to classify events by source when they have the same sourcetpye?

sdaniels — Fri, 27 Apr 2012 13:47:13 GMT

It's going to set the source based on the regex you provide.

Re: how to classify events by source when they have the same sourcetpye?

fernandoandre — Fri, 27 Apr 2012 14:25:43 GMT

I wasn't sure about "::$1" but nevermind.

Good example. Thank you for your assistance.

Re: how to classify events by source when they have the same sourcetpye?

yannK — Fri, 27 Apr 2012 14:58:00 GMT

Without specifications, the FORMAT goes directly to the _raw field.

But if you are using it to modify a metadata field (host, source, index, etc....) then you have to use the format
source::mysource (for a static source)
or in the case or a dynamic regex match

REGEX=blahblah(mymatch)blahblah
FORMAT=myfield::$1
the $1 $2, etc will be the different matches in the Regex.