how to classify events by source when they have the same sourcetpye?

[fernandoandre]

Hi

I'm indexing a file which is being written by a syslog process (therefore I defined the sourcetype=syslog) and this has events from different sources(origins). However the events are very similar and I want to tell Splunk how to "tag" this events by source at index time. My problem is not separate the events to different indexes but rather just classify them as different sources using a regex.

I have in inputs.conf:

[monitor:///data/syslog_cache]
index = my_index
sourcetype = syslog

I 'm thinking that I could do this in props.conf and transforms.conf but it doesn't work.

props.conf

[syslog]
EXTRACT-mySource1 = regex1
TRANSFORMS-set = myset1    
EXTRACT-mySource2 = regex2
TRANSFORMS-set = myset2

transforms.conf

[myset1]
source = mysource1

[myset2]
source = mysouce2

I'm probably confusing the goal behind this stanzas and attributes and I would appreciate any help.

[yannK · Accepted solution]

Hi, it seems that you have the pieces, but in the wrong order.

props.conf

[syslog]

TRANSFORMS-set = myset1,myset2, myset3

 # the transforms will apply in the order, and the last one can possibly replace the modifications from the previous ones.

transforms.conf

[myset1]

SOURCE_KEY = _raw

 # optional, this is the default

REGEX = 

DEST_KEY = MetaData:Source

FORMAT = source::$1

 # beware the caps are important, no caps for "source" in format, initial cap for "Source" and "Metadata" in Dest_key

[myset2]
# etc...

[yannK]

Without specifications, the FORMAT goes directly to the _raw field.

But if you are using it to modify a metadata field (host, source, index, etc....) then you have to use the format
source::mysource (for a static source)
or in the case or a dynamic regex match

REGEX=blahblah(mymatch)blahblah
FORMAT=myfield::$1
the $1 $2, etc will be the different matches in the Regex.

[fernandoandre]

I wasn't sure about "::$1" but nevermind.

Good example. Thank you for your assistance.

[sdaniels]

It's going to set the source based on the regex you provide.

[fernandoandre]

What's the specific role of "FORMAT = source::$1" ? Don't quite understand this part.