topic Re: How to write a regular expression to extract key value pairs from my sample event? in Splunk Search

How to write a regular expression to extract key value pairs from my sample event?

rajgowd1 — Tue, 29 Sep 2020 11:51:47 GMT

Hi,

We have events which contain key value pairs separated by a colon :.
Here is the sample event:

<6>2016-11-22T16:29:37Z v26l5klvoo3 doppler[21]: {"cf_app_id":"19351f6f-d125-4322-88ed-926e8f578e16","cf_app_name":"nam-rds-gpu-drag-payment-mq","cf_org_id":"d58e8896-6150-4f89-806b-c28464ba7e22","cf_org_name":"NAM-US-verizon","cf_origin":"firehorse","cf_space_id":"52223ed6-a3c5-4955-a6f8-be6c9649e215","cf_space_name":"RDS-DEV1","event_type":"LogMessage","level":"info","message_type":"OUT","msg":"[2016-11-22 16:29:37,017] DiscoveryClient-CacheRefreshExecutor-0 (DiscoveryClient.java:1064) DEBUG - Got delta update with apps hashcode UP_45_","origin":"rep","source_instance":"0","source_type":"APP","time":"2016-11-22T16:29:37Z","timestamp":1479832177017862007}

Key value pairs are with in curly brackets { }
Example: cf_space_name":"RDS-DEV1" this is one key value pair

Can you please help us to write a regular expression for the above sample event.

Re: How to write a regular expression to extract key value pairs from my sample event?

sundareshr — Tue, 22 Nov 2016 16:58:15 GMT

Try this

REGEX = "([^"]+)":"([^"]+)
FORMAT = $1:$2

Re: How to write a regular expression to extract key value pairs from my sample event?

rajgowd1 — Tue, 22 Nov 2016 17:31:28 GMT

Hi Sundaresh,
when i try index=myindex | rex "([^"]+)":"([^"]+)
getting error like Mismatched ']'. but the same expression is working in regex101

am i missing any here?

Re: How to write a regular expression to extract key value pairs from my sample event?

ppablo — Tue, 22 Nov 2016 18:13:44 GMT

Apologies @sundareshr and @rajgowd1 I accidentally clicked the Accept button. I unaccepted it since this question is still unresolved.

Re: How to write a regular expression to extract key value pairs from my sample event?

sundareshr — Tue, 22 Nov 2016 18:16:22 GMT

If you're using it in rex, you'll need to escape the quote. Try this

... | rex "\"(?<key>[^\"]+)\":\"(?<value>[^\"]+)" max_match=0

With this, key and value will be mvfields, to access a specific value, you will have to use mvindex(key, n) etc. I would not recommed this approach.

The other thing you could try is the extract command

... | extract kvdelim=":" pairdelim=","

This may work better for your case

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Extract

Re: How to write a regular expression to extract key value pairs from my sample event?

rajgowd1 — Tue, 22 Nov 2016 18:26:54 GMT

thank you.you are genius.regular expression and extract both are working as expected

Re: How to write a regular expression to extract key value pairs from my sample event?

sshelly_splunk — Tue, 22 Nov 2016 19:56:58 GMT

Try this one:

REGEX= (?ism)"(\w+)":"([^"]+)|timestamp":(?P\d+)
FORMAT = $1:$2