topic Re: Extracting an unknown number of fields with rex command? in Splunk Search

How to extract an unknown number of fields with rex command?

jbrenner — Fri, 11 Nov 2022 21:13:59 GMT

Let's say I have data in an event that looks like this:

NAME: John NAME: Mary NAME: Sue

Assuming I have no idea how many names will exist in the event, is it possible to use the rex command to parse out all the names and display them in separate fields?

Thanks,

Jonathan

Re: Extracting an unknown number of fields with rex command?

richgalloway — Fri, 11 Nov 2022 20:59:22 GMT

You can use the rex command with the max_match option to match an unpredictable number of names.

| rex max_match=0 "NAME: (?<Name>\S+)"

It will, however, put all of the matches into the single (multi-value) field called "Name". You then can use the mvexpand command to put each Name value into a separate event. That's not exactly "separate fields", though, is it?

What exactly do you mean by "separate fields"? What should the final result look like?

Re: Extracting an unknown number of fields with rex command?

yuanliu — Sat, 12 Nov 2022 03:46:40 GMT

I suspect that the OP meant separate events rather than separate fields, like

| rex max_match=0 "NAME: (?<Name>\S+)" | mvexpand Name

But to humor the literal interpretation, you could do something like

| rex max_match=0 "NAME: (?<Name>\S+)" | foreach 1 2 3 4 5 6 7 8 9 10 ``` more than mvcount(Name) ``` [eval "Name-<<FIELD>>" = mvindex(Name, <<FIELD>> - 1)]

This uses a side effect of SPL's handling of null assignments. (I wish the new multivalue foreach's <<ITEM>> could be used on the left-hand side. It cannot.)

Re: Extracting an unknown number of fields with rex command?

bowesmana — Sun, 13 Nov 2022 22:42:54 GMT

Another "technical" solution for making new fields is this, which first makes the name of the field, the name from the event, then uses streamstats to define the Name suffix, so doesn't have a foreach limitation.

| makeresults | eval _raw="NAME: John NAME: Mary NAME: Sue NAME: Jack NAME: Jill NAME: Peter NAME: Susan" | rex max_match=0 "NAME: (?<Name>\S+)" | mvexpand Name | streamstats c | eval x_{Name}=c | stats values(x_*) as * | foreach * [ eval "Name_{<<FIELD>>}"="<<FIELD>>" ] | fields Name_*

I say "technical", as this type of solution is rarely a real-life solution where mvexpand is an option in a large data set.