Splunk Search

How to write a regular expression to extract key value pairs from my sample event?

rajgowd1
Communicator

Hi,

We have events which contain key value pairs separated by a colon :.
Here is the sample event:

<6>2016-11-22T16:29:37Z v26l5klvoo3 doppler[21]: {"cf_app_id":"19351f6f-d125-4322-88ed-926e8f578e16","cf_app_name":"nam-rds-gpu-drag-payment-mq","cf_org_id":"d58e8896-6150-4f89-806b-c28464ba7e22","cf_org_name":"NAM-US-verizon","cf_origin":"firehorse","cf_space_id":"52223ed6-a3c5-4955-a6f8-be6c9649e215","cf_space_name":"RDS-DEV1","event_type":"LogMessage","level":"info","message_type":"OUT","msg":"[2016-11-22 16:29:37,017] DiscoveryClient-CacheRefreshExecutor-0 (DiscoveryClient.java:1064) DEBUG - Got delta update with apps hashcode UP_45_","origin":"rep","source_instance":"0","source_type":"APP","time":"2016-11-22T16:29:37Z","timestamp":1479832177017862007}

Key value pairs are with in curly brackets { }
Example: cf_space_name":"RDS-DEV1" this is one key value pair

Can you please help us to write a regular expression for the above sample event.

0 Karma
1 Solution

sundareshr
Legend

Try this

REGEX = "([^"]+)":"([^"]+)
FORMAT = $1:$2

View solution in original post

0 Karma

sshelly_splunk
Splunk Employee
Splunk Employee

Try this one:

REGEX= (?ism)"(\w+)":"([^"]+)|timestamp":(?P\d+)
FORMAT = $1:$2

0 Karma

sundareshr
Legend

Try this

REGEX = "([^"]+)":"([^"]+)
FORMAT = $1:$2
0 Karma

ppablo
Retired

Apologies @sundareshr and @rajgowd1 I accidentally clicked the Accept button. I unaccepted it since this question is still unresolved.

0 Karma

rajgowd1
Communicator

Hi Sundaresh,
when i try index=myindex | rex "([^"]+)":"([^"]+)
getting error like Mismatched ']'. but the same expression is working in regex101

am i missing any here?

0 Karma

sundareshr
Legend

If you're using it in rex, you'll need to escape the quote. Try this

... | rex "\"(?<key>[^\"]+)\":\"(?<value>[^\"]+)" max_match=0

With this, key and value will be mvfields, to access a specific value, you will have to use mvindex(key, n) etc. I would not recommed this approach.

The other thing you could try is the extract command

... | extract kvdelim=":" pairdelim=","

This may work better for your case

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Extract

rajgowd1
Communicator

thank you.you are genius.regular expression and extract both are working as expected

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...