topic Re: props.conf field extraction in Knowledge Management

props.conf field extraction

nikkkc — Thu, 28 Jul 2016 13:32:26 GMT

Hi,
i try to extract a field in props.conf on search head/indexer. Data comes from UF.

props.conf
[mysyslog]
EXTRACT-level = "var/log/remote/smg/mail\d+/\w+/(?[^/]*)/" in source

source: /var/log/remote/smg/mail01/mail/info/xxxxx.log

the regex work´s in search:
....| rex field=source "var/log/remote/smg/mail\d+/\w+/(?[^/]*)/"

but not in props.conf??
Why? i tried with quotes and without quotes....

Re: props.conf field extraction

nikkkc — Thu, 28 Jul 2016 13:54:09 GMT

xxs Security deletes some characters...

Hi,
i try to extract a field in props.conf on search head/indexer. Data comes from UF.

props.conf
[mysyslog]
EXTRACT-level = "var/log/remote/smg/mail\d+/\w+/(?<level>[^/]*)/" in source

source: /var/log/remote/smg/mail01/mail/info/xxxxx.log

the regex work´s in search:
....| rex field=source "var/log/remote/smg/mail\d+/\w+/(?<level>[^/]*)/"

but not in props.conf??
Why? i tried with quotes and without quotes....

Re: props.conf field extraction

twinspop — Thu, 28 Jul 2016 14:00:04 GMT

Removed: Wrong headed answer about EXTRACT vs REPORT

This was likely the cause of your problems: No quotes around the regex, in either case.

My recommendation:

props.conf:

[mysyslog]
REPORT-level = extract_level

transforms.conf

[extract_level]
SOURCE_KEY = source
REGEX = var/log/remote/smg/mail\d+/\w+/([^/]*)/
FORMAT = mylevel::$1

Re: props.conf field extraction

pradeepkumarg — Thu, 28 Jul 2016 14:01:11 GMT

You definitely don't need quotes. verify your updated props.conf is on your intended search head. you can also check this with the btool command

./splunk cmd btool props list

Re: props.conf field extraction

pradeepkumarg — Thu, 28 Jul 2016 14:05:14 GMT

EXTRACT is not index time field extractions. Check below from props.conf documentation

Use the TRANSFORMS field extraction type to create index-time field
extractions. Use the REPORT or EXTRACT field extraction types to create
search-time field extractions.

Re: props.conf field extraction

twinspop — Thu, 28 Jul 2016 14:09:30 GMT

Ah crap, you're right. Too early in the morning. 🙂

Re: props.conf field extraction

ddrillic — Thu, 28 Jul 2016 14:17:11 GMT

We learned in class the following -

Use extraction directives, EXTRACT and REPORT in props.conf

EXTARCT (inline extraction) is defined in props.conf as standalone
REPORT (field transform) is defined in transform.conf and invoked from props.conf

Re: props.conf field extraction

nikkkc — Fri, 29 Jul 2016 05:56:55 GMT

first i want to say thank you.
still one question: i do not need to specify the field in regex? like ?<mylevel>
OK, if i specify the field then i do not need the line: FORMAT = mylevel
right?

Anyway, i did a | extract reload=t
but still no new filed in my search gui

Re: props.conf field extraction

nikkkc — Fri, 29 Jul 2016 06:19:22 GMT

...i do not want to waste more time for this, does it make a differnece to use the rex in search query or to define in props and transforms conf?? because it work´s in search query

Re: props.conf field extraction

nikkkc — Fri, 29 Jul 2016 09:24:01 GMT

after restart splunk it works.