topic Re: Extraction in props.conf not working in Splunk Enterprise

Extraction in props.conf not working

manuarora — Wed, 18 Aug 2010 20:30:34 GMT

I have following inputs.conf

[script://$SPLUNK_HOME/etc/apps/mck-perflog-aix/bin/lsvgdetails.sh]
index = mck-perflog
sourcetype = lsvg_detail
interval = 60

Script returns output as:-
rootvg,64,559,199,360

I am using following in props.conf

[lsvg_detail]
EXTRACT-common = (?<vg_name>[^\,]+),(?<pp_size>[^\,]+),(?<total_pp>[^\,]+),(?<free_pp>[^\,]+),(<used_pp>[^\,]+)

but it is not extracting field, I do not have any transforms.conf

Can you please help

Re: Extraction in props.conf not working

Lowell — Wed, 18 Aug 2010 20:42:34 GMT

Any reason why you aren't using delimiter based extractions? I think that would be simpler in your case.

That said, the bug seems to be in your regex. Try this:

EXTRACT-common = ^(?<vg_name>[^\,]+),(?<pp_size>[^\,]+),(?<total_pp>[^\,]+),(?<free_pp>[^\,]+),(?<used_pp>[^\,]+)

You were missing a "?" at the end (used_pp). I also added a leading "^" which will force your regex to start at the beginning of the line; which is better than letting the regex engine guess.

I didn't change this, but you should note that your entire regex will fail to match if any of the fields you have defined are missing. That is [^\,]+ vs [^\,]*. You may or may not want this. Also, you don't need the backslash before your commas.

To use a delimiter based extraction, you can do the following:

props.conf:

[lsvg_detail]
SHOULD_LINEMERGE = False
DATETIME_CONFIG = CURRENT
REPORT-fields = lsvg_detail-fields

transforms.conf:

[lsvg_detail-fields]
DELIMS = ","
FIELDS = "vg_name", "pp_size", "total_pp", "free_pp", "used_pp"

In my option this is much simpler to understand and maintain in the future; and it may even be slightly faster than the regex approach (not that you're likely to notice the difference.) So unless you have a complex field extraction situation (i.e optional columns or very weird quoting logic) the I recommend sticking to the delimiter based approach. (And that's coming from someone who really enjoys writing and tweaking regular expressions.)

Re: Extraction in props.conf not working

manuarora — Mon, 28 Sep 2020 09:16:24 GMT

Hi it gives me output like this
splunkdvg,128,399,6,393
appvg,128,478,357,121
rootvg,64,559,199,360
but when I use above format it adds VG name to used_pp, I think I need to use LINE_BREAKER but do not know what should be value of line breaker

Re: Extraction in props.conf not working

manuarora — Thu, 19 Aug 2010 23:02:26 GMT

I am using multiple unix echo to display text i.e
echo "splunkdvg,128,399,6,393"
echo "appvg,128,478,357,121"

Re: Extraction in props.conf not working

Lowell — Mon, 28 Sep 2020 09:16:26 GMT

Your problem is probably that you don't a timestamp in your event. Splunk breaks by default after a timestamp. You don't want to change LINE_BREAKER in this case (or hardly ever), you simply want to add SHOULD_LINEMERGE = False in props. (I've updated my example above to reflect this.)