topic Re: Is there a way to force rex to be evaluated before mv functions? in Splunk Search

Is there a way to force rex to be evaluated before mv functions?

narduk — Thu, 23 Oct 2014 18:51:34 GMT

I am having a problem extracting multivalued fields. I think it's because this particular field is quoted.

ids=\"XXX-404994280,XXX-404993710,XXX-335205060,XXX-404991340,XXX-335203510\"

The following search: index=app_logs env=prod | makemv delim="," ids | mvexpand ids

Does not yield the expected results of 5 new events.

It seems like this is a bug in the way Splunk evaluates multi valued fields that is agitated by the slash and the quote so I was trying to get around this problem by removing the

\"

It seems like Splunk must run the rex commands after the mv commands. Is there any way to force it to run rex first? Is there any documentation on the order of operations of the splunk commands?

Re: Is there a way to force rex to be evaluated before mv functions?

somesoni2 — Thu, 23 Oct 2014 19:01:36 GMT

Does the raw data contains ids field enclosed within \" or just "?

Re: Is there a way to force rex to be evaluated before mv functions?

narduk — Thu, 23 Oct 2014 19:03:22 GMT

The raw data contains the \". I'll edit the question to reflect that. I see what you did there.

Re: Is there a way to force rex to be evaluated before mv functions?

martin_mueller — Thu, 23 Oct 2014 22:43:48 GMT

Maybe I misunderstand your question, but you mention rex commands not running at the point in the search pipeline you want them to... I see no rex command in your search?

Re: Is there a way to force rex to be evaluated before mv functions?

narduk — Thu, 23 Oct 2014 23:56:44 GMT

Ok, so here's the search:

index=app_logs env=poi-prod rapIds | rex field=_raw mode=sed "s/\\\\\"//g" | makemv delim="," ids | mvexpand ids

and the pertinent piece of the event that I hope to separate into 5 events.

ids=\"XXX-404994280,XXX-404993710,XXX-335205060,XXX-404991340,XXX-335203510\"

Re: Is there a way to force rex to be evaluated before mv functions?

jrodman — Fri, 24 Oct 2014 05:02:36 GMT

The problem is that you are using rex to modify the text of the event, _raw, but then your makemv is targetting the field ids. If you want rex to have an effect upon the makemv, you will need to use it to modify the ids field instead.

Re: Is there a way to force rex to be evaluated before mv functions?

martin_mueller — Fri, 24 Oct 2014 13:17:12 GMT

Indeed, for example like this:

... | rex field=ids mode=sed "s/\\\\"//g" | makemv ...

Re: Is there a way to force rex to be evaluated before mv functions?

narduk — Fri, 24 Oct 2014 14:29:13 GMT

Here's the problem. The field ids, when I look at it in the field viewer is this:

\

So the problem is that Splunk terminates the field at the \ instead of continuing on until the space. I think my thought process here is, if I can remove the troubled \ " characters and get Splunk to reevaluate the field, I would be ok.

Ideas?

Re: Is there a way to force rex to be evaluated before mv functions?

martin_mueller — Fri, 24 Oct 2014 18:11:59 GMT

Actually the best solution is to write your own field extraction that is aware of the backslashes instead of trying to nudge a failing key-value extraction back to life.

Something like this:

\bids=\\?"?(?<my_ids>[^\\"]+)

Re: Is there a way to force rex to be evaluated before mv functions?

narduk — Fri, 24 Oct 2014 18:31:13 GMT

Nailed it. Excellent work around. I created a new field: aaIds. Prefixing my created fields with aa is a trick I use to get them to show up at the top of the fields list. With the new extracted field I was able to expand as I expected.

Thanks, Martin!

Re: Is there a way to force rex to be evaluated before mv functions?

martin_mueller — Fri, 24 Oct 2014 19:27:49 GMT

Cool. I've converted this to an answer so you can mark that as the solution.