topic Re: How can I run the extract command on field extracted by the rex command in Splunk Search

How can I run the extract command on field extracted by the rex command

seomaniv — Thu, 21 Mar 2019 23:59:35 GMT

Cog in a larger machine, I have asked my Splunk team to improve the parsing on some of our logs, but it hasn't happened yet, so I need a way to parse complex events in my search.

I have run my raw event(s) through rex and gotten several fields of interest (eg. field1, field2, field3, field4). Field 3 is a json, which I can then run through spath to get the key:value pairs, but field4 is a different series of key:value pairs (formatted as field-name=[fieldValues]).

My question is, I run my rex, then I pipe, then I run spath, then... how do I run extract?

Example event:

2019-03-20 19:25:16.010 Container: 32 Fred {json}, {Date=[some date and time and zone], Another-field=[values=may have; subvalues=with spaces; but no internal closing bracket], and-so-on=[until the last field]}

Note: I am trying to keep it all simple other than field4, which is the basis of my question; I am not having a problem with rex, so don't focus on my rex expressions (I won't even put them), but I can easily extract the other fields with rex, as well as the json fields with the spath:

rex field=_raw "Container: (?<field1>AppropriateRex) (?<field2>AppropriateRex) (?<field3>AppropriateRex),\{(?<field4>AppropriateRex)\}" | spath input=field3

so I would get:

field1=32
field2=Fred
field3={json}
field4

Date=[some date and time and zone], Another-field=[values=may have; subvalues=with spaces; but no internal closing bracket], and-so-on=[until the last field]

and let's say all 50 of the fields from my json are properly extracted via spath. How do I add an "extract" command to get the key:value pairs from field4? I'd like to:

| extract input=field4 ...

but extract doesn't have an "input" parameter.

Thank's in advance for your help.

Re: How can I run the extract command on field extracted by the rex command

sduff_splunk — Fri, 22 Mar 2019 08:23:49 GMT

You can try renaming the field you want to extract from to _raw , then run extract

rex field=_raw "Container: (?<field1>AppropriateRex) (?<field2>AppropriateRex) (?<field3>AppropriateRex),\{(?<field4>AppropriateRex)\}" | spath input=field3 | rename field4 AS _raw | extract

Re: How can I run the extract command on field extracted by the rex command

seomaniv — Mon, 25 Mar 2019 17:18:06 GMT

Thank you. This allows the proper extraction of the new fields, as well as keeping the previously extracted fields available, but the event itself is reduced to field4 (I assume because we renamed it as _raw). Is there a way to extract the fields and still display the entire event? For my current needs, I am making a table out of the data to display to non-tech users, so the raw data is not terribly important, but I fear at some point in the future I will regret saying that ...

Re: How can I run the extract command on field extracted by the rex command

sduff_splunk — Fri, 29 Mar 2019 04:42:50 GMT

rex field=_raw "Container: (?AppropriateRex) (?AppropriateRex) (?AppropriateRex),\{(?AppropriateRex)\}" | spath input=field3 |eval orig_raw = _raw | rename field4 AS _raw | extract | eval _raw = orig_raw

Re: How can I run the extract command on field extracted by the rex command

seomaniv — Tue, 02 Apr 2019 22:37:45 GMT

Thank you, sduff! That worked perfectly!