I'm indexing a file which is being written by a syslog process (therefore I defined the sourcetype=syslog) and this has events from different sources(origins). However the events are very similar and I want to tell Splunk how to "tag" this events by source at index time. My problem is not separate the events to different indexes but rather just classify them as different sources using a regex.
I have in inputs.conf:
[monitor:///data/syslog_cache] index = my_index sourcetype = syslog
I 'm thinking that I could do this in props.conf and transforms.conf but it doesn't work.
[syslog] EXTRACT-mySource1 = regex1 TRANSFORMS-set = myset1 EXTRACT-mySource2 = regex2 TRANSFORMS-set = myset2
[myset1] source = mysource1 [myset2] source = mysouce2
I'm probably confusing the goal behind this stanzas and attributes and I would appreciate any help.
Hi, it seems that you have the pieces, but in the wrong order.
TRANSFORMS-set = myset1,myset2, myset3
# the transforms will apply in the order, and the last one can possibly replace the modifications from the previous ones.
SOURCEKEY = _raw
# optional, this is the default
REGEX = <yourregex>
DESTKEY = MetaData:Source
FORMAT = source::$1
# beware the caps are important, no caps for "source" in format, initial cap for "Source" and "Metadata" in Destkey
What's the specific role of "FORMAT = source::$1" ? Don't quite understand this part.
I wasn't sure about "::$1" but nevermind.
Good example. Thank you for your assistance.
Without specifications, the FORMAT goes directly to the _raw field.
But if you are using it to modify a metadata field (host, source, index, etc....) then you have to use the format
source::mysource (for a static source)
or in the case or a dynamic regex match
the $1 $2, etc will be the different matches in the Regex.