topic Re: dynamically assign sourcetype on folder in Getting Data In

dynamically assign sourcetype on folder

fredkaiser — Mon, 15 Feb 2016 01:24:00 GMT

I'll like to assign the sourcetype on the folder the logs are sitting in

What I have

File location pick up by forwarder

C:\Program Files (x86)\License\Current\test\filename.log

props.conf

[source::C:\\Program Files (x86)\\License\\Current\\*\\*.log] 
TRANSFORMS-set_sourcetype = set_sourcetype_from_log_subdir

Transforms.conf

[set_sourcetype_from_log_subdir]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
REGEX = .+\\(.+)\\[^]*
FORMAT = sourcetype::$1

Splunk Returns sourcetype as filename when index

but I want is the folder not the file name so it should return test

Re: dynamically assign sourcetype on folder

MuS — Mon, 15 Feb 2016 01:42:51 GMT

Hi fredkeiser,

Your regex is missing the double escaped backslashes and you can test it with Splunk's internal regex functions:

splunk cmd pcregextest test_str="C:\Program Files (x86)\LIC\Current\test\filename.log" mregex=".+\\(.+)\\[^\\]*"
returns:

Original Pattern: '.+\(.+)\[^\]*'
Expanded Pattern: '.+\(.+)\[^\]*'
ERROR: Regex: unmatched parentheses

But if you double escape the backslashes it works:

splunk cmd pcregextest test_str="C:\Program Files (x86)\LIC\Current\test\filename.log" mregex=".+\\\(.+)\\\[^\\\]+"
returns:

Original Pattern: '.+\\(.+)\\[^\\]+'
Expanded Pattern: '.+\\(.+)\\[^\\]+'
Regex compiled successfully. Capture group count = 1. Named capturing groups = 0.
SUCCESS - match against: 'C:\Program Files (x86)\LIC\Current\test\filename.log'

#### Capturing group data ##### 
Group |            Name | Value
--------------------------------------
    1 |                 | test

Hope this helps ...

cheers, MuS

Re: dynamically assign sourcetype on folder

fredkaiser — Mon, 15 Feb 2016 02:11:49 GMT

I'll try that as well and still returns the file name as the sourcetype. even when you test your regex here http://www.regexr.com/ it shows it should work. But for some reason it doesn't work with Splunk

Re: dynamically assign sourcetype on folder

MuS — Mon, 15 Feb 2016 02:21:50 GMT

Ahh, sorry my bad ! I'll update the answer ....

Re: dynamically assign sourcetype on folder

MuS — Mon, 15 Feb 2016 02:25:56 GMT

update done

Re: dynamically assign sourcetype on folder

fredkaiser — Wed, 17 Feb 2016 01:00:49 GMT

found the issue was to do with are Props.conf

Props.conf

[source::C:\Program Files (x86)\\Current\\*\\*.log]

should be

[source::...\\Current\\*\\*.log]

everything else was right.

Re: dynamically assign sourcetype on folder

fredkaiser — Wed, 17 Feb 2016 01:06:14 GMT

Thanks for the help MuS