topic Re: Can I use regex to assign a sourcetype? in Splunk Search

Can I use regex to assign a sourcetype?

linwqg — Tue, 29 Sep 2020 15:55:21 GMT

Hello. I new to regex and have been trying to understand how it works.

Let say i have a log containing strings of information. I am to index it to splunk and assign a sourcetype to it via props.conf and transform.conf. Am i suppose to use regex to match a string, and if match, proceed to assign sourcetype?

1- Example, log contents as following:
"This log belong to ABC"

2 - In transforms.conf:
[assign_sourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = (ABC|ABC)
FORMAT = sourcetype::ABC

Anyway, the above regex does not work. Any help much appreciated.

Re: Can I use regex to assign a sourcetype?

gcusello — Mon, 25 Sep 2017 10:01:30 GMT

Hi linwqg,
if you're new to regex, you could follow two ways:

use the Splunk Extraction field web interface,
put your example in regex101.com and find the correct regex using this test url.

Anyway, I'm not sure that you can assign a sourcetype using a regex, because sourcetype is a field that identify a data flow and all the following knowledge objects (fields, eventtypes, ...) are related to sourcetype, so if you have dynamic sourcetype,s how your knowledge Objects can follow sourcetypes?

Bye.
Giuseppe

Re: Can I use regex to assign a sourcetype?

mwdbhyat — Mon, 25 Sep 2017 10:14:21 GMT

Hi,

You specify the sourcetype at input time(it will be more efficient) in inputs.conf..

So you would have your monitor stanza [monitor this file] under that you put sourcetype = mysourcetype.

http://docs.splunk.com/Documentation/SplunkCloud/6.6.1/Data/Monitorfilesanddirectorieswithinputs.conf

There is no need for regex unless you are trying to extract multiple sourcetypes from a single log or something like that

Re: Can I use regex to assign a sourcetype?

logloganathan — Mon, 19 Mar 2018 11:48:25 GMT

Please find the regex " (?ms).to(?.)" for the example you provided "This log belong to ABC"
similarly you have to proceed using the online tool https://regex101.com/

Re: Can I use regex to assign a sourcetype?

logloganathan — Mon, 19 Mar 2018 15:49:39 GMT

any update? is this helpful?

Re: Can I use regex to assign a sourcetype?

skoelpin — Mon, 19 Mar 2018 16:30:39 GMT

Do NOT do it this way. I've seen many environments where they create sourcetypes willy-nilly and wonder why it takes forever to onboard data. Every time you create a unique sourcetype, you need to write base configs which tell the indexers how to break the events and how to read the timestamp. The best approach is to use the least amount of sourcetypes and have a standard sourcetype for each data format. Then use eventtypes to differentiate between apps and environments like you were with sourcetypes