topic Re: How do I extract these fields from my data using rex? in Splunk Search

How do I extract these fields from my data using rex?

IRHM73 — Wed, 30 Sep 2015 08:25:01 GMT

Hi,

I wonder whether someone could help me please.

I have a field called detail.cid-repsonse which looks like the following:

[{"name":{"current":{"firstName":"JOHN","lastName":"SMITH"}},"ids":{"sut":"1234567890","nino":"AA111111A"},"dateOfBirth":"26121973"}]

From this I need to create new fields and extract the following data:

First Name
Last Name
Sut
NINO
DOB

I just wondered whether someone may be able to offer some guidance on how I may go about this please.

Any help would be greatly appreciated.

Many thanks and kind regards

Chris

Re: How do I extract these fields from my data using rex?

tom_frotscher — Wed, 30 Sep 2015 09:51:29 GMT

Hi,

you simply can do this with the rex command. You can restrict the rex command to one field with the field parameter. Here an example for the first name:

... | rex field="detail.cid-repsonse" "\"firstName\":\"(?<firstName>[^\"]+)" | ...

I created a user everywhere example, which means, you can copy the follwoing search and paste it to your splunk search line and it will work. This can give you an idea how things work:

| stats count | eval "detail.cid-repsonse"="[{\"name\":{\"current\":{\"firstName\":\"JOHN\",\"lastName\":\"SMITH\"}},\"ids\":{\"sut\":\"1234567890\",\"nino\":\"AA111111A\"},\"dateOfBirth\":\"26121973\"}]" | rex field="detail.cid-repsonse" "\"firstName\":\"(?<firstName>[^\"]+)" | rex field="detail.cid-repsonse" "\"lastName\":\"(?<lastName>[^\"]+)" | rex field="detail.cid-repsonse" "\"sut\":\"(?<sut>[^\"]+)" | table "detail.cid-repsonse" firstName lastName sut

By the way, it looks like you have valid json in your field, so you might also be able to use the spath command: http://docs.splunk.com/Documentation/Splunk/6.2.5/SearchReference/Spath

In your case just append this to your search:

| spath input="detail.cid-repsonse"

and you will get new fields with your needed data.

Greetings

Tom

Re: How do I extract these fields from my data using rex?

IRHM73 — Wed, 30 Sep 2015 11:20:24 GMT

Hi Tom, this is great and works a treat.

Thank you for taking the time to reply to my post.

Kind Regards and thanks

Chris

Re: How do I extract these fields from my data using rex?

dmaislin_splunk — Wed, 30 Sep 2015 11:47:36 GMT

Just use this technique:

http://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Extractfieldsfromfileheadersatindextime

Forwarder props.conf entry for the specific sourcetype.
INDEXED_EXTRACTIONS=JSON

Fields are fed to indexers from the forwarder and searches will be much faster as a result.

Re: How do I extract these fields from my data using rex?

IRHM73 — Wed, 30 Sep 2015 11:50:35 GMT

Hi @dmaislin, thank you for taking the time to reply to my post. I'm very new to Splunk, so your solution may be a little over my head, but I really appreciate you highlighting something which I will no doubt be able to use in the future.

Many thanks and kind regards

Chris

Re: How do I extract these fields from my data using rex?

tom_frotscher — Wed, 30 Sep 2015 11:50:38 GMT

I think he does not have json only. Just the field he mentioned in his questions is json. If the complete event is json, your anser might be the even better option.

Re: How do I extract these fields from my data using rex?

dmaislin_splunk — Wed, 30 Sep 2015 11:57:13 GMT

No problem. If the logged events are JSON, this technique is probably the simplest approach as all of your fields will be present without requiring any extra field extraction work.

Re: How do I extract these fields from my data using rex?

dmaislin_splunk — Wed, 30 Sep 2015 11:58:55 GMT

Yes, it speeds up search and offloads the indexers from having to perform line-breaking and timestamp recognition tasks too.