Splunk Search

Is there a way to force rex to be evaluated before mv functions?

narduk
Explorer

I am having a problem extracting multivalued fields. I think it's because this particular field is quoted.

ids=\"XXX-404994280,XXX-404993710,XXX-335205060,XXX-404991340,XXX-335203510\"

The following search: index=app_logs env=prod | makemv delim="," ids | mvexpand ids

Does not yield the expected results of 5 new events.

It seems like this is a bug in the way Splunk evaluates multi valued fields that is agitated by the slash and the quote so I was trying to get around this problem by removing the

\"

It seems like Splunk must run the rex commands after the mv commands. Is there any way to force it to run rex first? Is there any documentation on the order of operations of the splunk commands?

Tags (3)
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Actually the best solution is to write your own field extraction that is aware of the backslashes instead of trying to nudge a failing key-value extraction back to life.

Something like this:

\bids=\\?"?(?<my_ids>[^\\"]+)

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Actually the best solution is to write your own field extraction that is aware of the backslashes instead of trying to nudge a failing key-value extraction back to life.

Something like this:

\bids=\\?"?(?<my_ids>[^\\"]+)

martin_mueller
SplunkTrust
SplunkTrust

Cool. I've converted this to an answer so you can mark that as the solution.

0 Karma

narduk
Explorer

Nailed it. Excellent work around. I created a new field: aaIds. Prefixing my created fields with aa is a trick I use to get them to show up at the top of the fields list. With the new extracted field I was able to expand as I expected.

Thanks, Martin!

jrodman
Splunk Employee
Splunk Employee

The problem is that you are using rex to modify the text of the event, _raw, but then your makemv is targetting the field ids. If you want rex to have an effect upon the makemv, you will need to use it to modify the ids field instead.

martin_mueller
SplunkTrust
SplunkTrust

Indeed, for example like this:

... | rex field=ids mode=sed "s/\\\\"//g" | makemv ...
0 Karma

narduk
Explorer

Here's the problem. The field ids, when I look at it in the field viewer is this:

\

So the problem is that Splunk terminates the field at the \ instead of continuing on until the space. I think my thought process here is, if I can remove the troubled \ " characters and get Splunk to reevaluate the field, I would be ok.

Ideas?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Maybe I misunderstand your question, but you mention rex commands not running at the point in the search pipeline you want them to... I see no rex command in your search?

0 Karma

narduk
Explorer

Ok, so here's the search:

index=app_logs env=poi-prod rapIds | rex field=_raw mode=sed "s/\\\\\"//g" | makemv delim="," ids | mvexpand ids

and the pertinent piece of the event that I hope to separate into 5 events.

ids=\"XXX-404994280,XXX-404993710,XXX-335205060,XXX-404991340,XXX-335203510\"

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Does the raw data contains ids field enclosed within \" or just "?

0 Karma

narduk
Explorer

The raw data contains the \". I'll edit the question to reflect that. I see what you did there.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...