topic How to extract multiple values in a single event into one multivalue field? in Splunk Search

How to extract multiple values in a single event into one multivalue field?

Ossian — Thu, 12 Mar 2015 20:13:38 GMT

This is something that I feel should be relatively simple, but no matter what I try I can't get the results I want. Basically I have a script to check local admins on systems and outputs the below as a single event:

Administrator
WORLD\Domain Admins
WORLD\Some.Person
WORLD\More Admins
WORLD\EvenMore Admins

I would like to get these all into a multivalue field called LocalAdmin. I've tried just about every suggestion I could find on here to generate multivalue fields, but nothing seems to work for me. Typically I either get just the first admin listed or a concatenation of all admins in a single value. I can't even get it to work with mvmake as I'm not sure if backslash n is a valid delim value.

I'm the splunk admin so I don't mind if this needs to be done at search time or index time. Thank you for any help provided, this has been driving me crazy as I've solved what seems to be much more daunting splunk challenges.

Update

This is still frustrating me, but at least I figured out what I was doing wrong with the inline regex. I forgot that rex defaults to a single match. The below rex does what I need but I cannot for the life of me get it to work in props and transforms.

rex max_match=1000 "(?<LocalAdmin>.*\n|.*$)"

Closest I'm able to come is the following in props and transforms:

[Script:LocalAdmins]
REPORT-LocalAdmin = mv_localadmin

[mv_localadmin]
REGEX = (.*\n|.*$)
FORMAT = LocalAdmin::$1
REPEAT_MATCH = true
MV_ADD = true

This actual does extract something, but I get all the admins in one value except for the last one

LocalAdmin= Administrator  WORLD\Domain Admins WORLD\Some.Person WORLD\More Admins
LocalAdmin= WORLD\EvenMore Admins

I've yet to figure out why and how the regex command acts so differently between the search and transforms. No matter how I try to edit that regex, I still get most admins in a single value even though the same regex work great in search.

Re: How to extract multiple values in a single event into one multivalue field?

ramdaspr — Wed, 18 Mar 2015 06:05:14 GMT

shouldnt the transforms.conf REGEX be

(?<LocalAdmin>.*\n|.*$)

Re: How to extract multiple values in a single event into one multivalue field?

tachifelix — Wed, 18 Mar 2015 08:55:07 GMT

try something like this:

 ....|makemv  delim = ":" tokenizer = (.*\n|.*$)  allowempty=t  administrator

administrator is your mutivalue field

Re: How to extract multiple values in a single event into one multivalue field?

Ossian — Wed, 25 Mar 2015 10:55:46 GMT

This issue is that I cannot get a field extraction to work properly, I can extract just fine in the search.

Re: How to extract multiple values in a single event into one multivalue field?

Ossian — Wed, 25 Mar 2015 11:18:13 GMT

Finally figured this out. Was filling out a support ticket on this "issue" and one of the answers they linked to was http://answers.splunk.com/answers/23321/field-regex-behaves-differently-for-field-extractions-and-for-rex.html. Apparently you need to add "(?m-s)" to keep "." from matching across newlines when using a regex in a transform. I just wish there was better documentation about the different behavior of regex's in different contexts within Splunk.

The following transform now works great for me:

[mv_localadmin]
REGEX = (?m-s)(.*\n|.*$)
FORMAT = LocalAdmin::$1
REPEAT_MATCH = true
MV_ADD = true