Is there a way to force rex to be evaluated before mv functions?

[narduk]

I am having a problem extracting multivalued fields. I think it's because this particular field is quoted.

ids=\"XXX-404994280,XXX-404993710,XXX-335205060,XXX-404991340,XXX-335203510\"

The following search: index=app_logs env=prod | makemv delim="," ids | mvexpand ids

Does not yield the expected results of 5 new events.

It seems like this is a bug in the way Splunk evaluates multi valued fields that is agitated by the slash and the quote so I was trying to get around this problem by removing the

\"

It seems like Splunk must run the rex commands after the mv commands. Is there any way to force it to run rex first? Is there any documentation on the order of operations of the splunk commands?

[martin_mueller · Accepted solution]

Actually the best solution is to write your own field extraction that is aware of the backslashes instead of trying to nudge a failing key-value extraction back to life.

Something like this:

\bids=\\?"?(?<my_ids>[^\\"]+)

[martin_mueller]

Cool. I've converted this to an answer so you can mark that as the solution.

[narduk]

Nailed it. Excellent work around. I created a new field: aaIds. Prefixing my created fields with aa is a trick I use to get them to show up at the top of the fields list. With the new extracted field I was able to expand as I expected.

Thanks, Martin!

[jrodman]

The problem is that you are using rex to modify the text of the event, _raw, but then your makemv is targetting the field ids. If you want rex to have an effect upon the makemv, you will need to use it to modify the ids field instead.

[martin_mueller]

Indeed, for example like this:

... | rex field=ids mode=sed "s/\\\\"//g" | makemv ...

[narduk]

Here's the problem. The field ids, when I look at it in the field viewer is this:

\

So the problem is that Splunk terminates the field at the \ instead of continuing on until the space. I think my thought process here is, if I can remove the troubled \ " characters and get Splunk to reevaluate the field, I would be ok.

Ideas?

[martin_mueller]

Maybe I misunderstand your question, but you mention rex commands not running at the point in the search pipeline you want them to... I see no rex command in your search?

[narduk]

Ok, so here's the search:

index=app_logs env=poi-prod rapIds | rex field=_raw mode=sed "s/\\\\\"//g" | makemv delim="," ids | mvexpand ids

and the pertinent piece of the event that I hope to separate into 5 events.

ids=\"XXX-404994280,XXX-404993710,XXX-335205060,XXX-404991340,XXX-335203510\"

[somesoni2]

Does the raw data contains ids field enclosed within \" or just "?

[narduk]

The raw data contains the \". I'll edit the question to reflect that. I see what you did there.