topic Re: How do I extract a field from my raw data using rex? in Splunk Search

How do I extract a field from my raw data using rex?

IRHM73 — Mon, 13 Jul 2015 06:15:24 GMT

Hi,

I wonder whether someone may be able to help me please.

I'm very new to using Splunk and most certainly to the rex command and regular expressions, so please bear with.

I'm trying to extract a nino field from my raw data which is in the following format "nino\":\"AB123456B\".

Could someone possibly tell me please how I may strip the actual nino number out of this line.

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 07:41:22 GMT

Hi IRHM73,

if your _raw events really contains the provided example , you can run this rex command:

your base reach here | rex field=_raw ":\\"(?<nino>\w+)\\""

This will match every match any word character [a-zA-Z0-9_] after :\" and puts is into the field called nino

Hope this helps ...

cheers, MuS

Re: How do I extract a field from my raw data using rex?

IRHM73 — Mon, 13 Jul 2015 07:51:33 GMT

Hi @MuS, thank you very much for taking the time to reply to my post.

Unfortunately I can't get this to work, because it returns the error message: Error in 'rex' command: Encountered the following error while compiling the regex ':(?w+)': Regex: unmatched parentheses.

Could you also tell me please, is it possible to search for the field 'nino' specifically because my raw data contains a number of fields with the same "fieldname":"fieldvalue" format.

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 08:28:02 GMT

Sorry my bad, try this:

your base reach here | rex field=_raw ":\\\"(?<nino>\w+)\\\""

or take a look at this answer to answer your second question http://answers.splunk.com/answers/214487/can-i-extract-a-field-with-a-regexed-dynamic-field.html 😉

Re: How do I extract a field from my raw data using rex?

IRHM73 — Mon, 13 Jul 2015 08:39:47 GMT

Hi, that's great thank you very much, but could you tell me please how I search for the 'nino' field.

Could you also tell me please, is it possible to search for the field 'nino' specifically because my raw data contains a number of fields with the same "fieldname":"fieldvalue" format.

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 08:48:35 GMT

Just add any search or stats command in the next search pipe like this:

your base reach here | rex field=_raw ":\\\"(?<nino>\w+)\\\"" | table nino

your base reach here | rex field=_raw ":\\\"(?<nino>\w+)\\\"" | search nino=AB*

Re: How do I extract a field from my raw data using rex?

IRHM73 — Mon, 13 Jul 2015 08:51:30 GMT

Hi @MuS, that's great, thank you.

Kind Regards

Chris

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 08:55:43 GMT

You're welcome 🙂

Re: How do I extract a field from my raw data using rex?

otman01 — Mon, 13 Jul 2015 09:00:22 GMT

Hi, every one,
how can we use rex field=_raw in an input field ? I couldn't make it correctley

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 09:03:35 GMT

I suggest to open a new question and provide more details

Re: How do I extract a field from my raw data using rex?

IRHM73 — Mon, 13 Jul 2015 09:21:52 GMT

Hi, @MuS, I'm really very sorry to trouble you with this again.

I've used the 'rex' expression you kindly provided, but unfortunately it doesn't extract the correct data into the table. I've also tried anothe field called 'middleNames' which has the same format as the NINO, so the expression is rex field=_raw ":"(?w+)"" | table middleNames, but it is still not extracting the field value data from the raw text.

Any ideas?

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 09:35:48 GMT

please provide some raw event for both fields, use the code mark up CTRL-K to encode any special characters ...

Re: How do I extract a field from my raw data using rex?

IRHM73 — Mon, 13 Jul 2015 09:58:53 GMT

Hi @Mus, thank you for coming back to me with this.

Unfortunately, due to it's sensitive data, I can't send you the raw data, but perhaps if you could explain, what you need I may be able to put something together.

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

woodcock — Mon, 13 Jul 2015 15:18:33 GMT

This should work:

... | rex "nino[\\\":]*(?<nino>[^\\\":]*)"

Re: How do I extract a field from my raw data using rex?

MuS — Mon, 13 Jul 2015 21:46:37 GMT

Hi there, it is currently not 100% clear if the event looks like this nino":"AB123456B" or like this nino\":\"AB123456B\" can you please confirm if it is either the first or the later one?

Re: How do I extract a field from my raw data using rex?

IRHM73 — Tue, 14 Jul 2015 06:44:51 GMT

Hi @MuS, thank you for coming back to me with this I really appreciate it. The format of the nino field was "nino\":\"AB123456B\". But not to worry I've been able to extrcat this using the solution by @woodcock.

Once again sincere thanks for your help.

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

IRHM73 — Tue, 14 Jul 2015 06:45:34 GMT

Hi @woodcock, thank you for taking the time to reply to my post, this works great.

Many thanks and kind regards

Chris

Re: How do I extract a field from my raw data using rex?

bablucho — Thu, 18 Oct 2018 13:42:15 GMT

Hi All,

I'm trying to achieve the something similar using regular expressions on raw data...

I want to extract the figures at the end of row ESS1. if some-one could help me extract the field and strip out the figures would be great!

ESCO - [57]
ESDC - [82]
ESEM - [170]
ESS1 - [2813]
ESVI - [706]
F421 - [30]

Thanks
Bablucho

Re: How do I extract a field from my raw data using rex?

MuS — Thu, 18 Oct 2018 19:04:22 GMT

Try this:

 ... | rex "ESS1 - \[(?<thatfield>[^\]]+)"

this will create a field called thatfield with the value 2813 based on your provided example.

cheers, MuS

Re: How do I extract a field from my raw data using rex?

bablucho — Fri, 19 Oct 2018 08:11:47 GMT

Thanks MuS I think this would work though I've just realised when raw data I pasted in didn't include the whites spaces...

ESCO - [57]
ESDC - [82]
ESEM - [170]
ESS1{14whitespaces}-{3whitespaces} [2813]
ESVI - [706]
F421 - [30]