Splunk Search

How to write a regular expression to extract key value pairs from my sample event?

rajgowd1
Communicator

Hi,

We have events which contain key value pairs separated by a colon :.
Here is the sample event:

<6>2016-11-22T16:29:37Z v26l5klvoo3 doppler[21]: {"cf_app_id":"19351f6f-d125-4322-88ed-926e8f578e16","cf_app_name":"nam-rds-gpu-drag-payment-mq","cf_org_id":"d58e8896-6150-4f89-806b-c28464ba7e22","cf_org_name":"NAM-US-verizon","cf_origin":"firehorse","cf_space_id":"52223ed6-a3c5-4955-a6f8-be6c9649e215","cf_space_name":"RDS-DEV1","event_type":"LogMessage","level":"info","message_type":"OUT","msg":"[2016-11-22 16:29:37,017] DiscoveryClient-CacheRefreshExecutor-0 (DiscoveryClient.java:1064) DEBUG - Got delta update with apps hashcode UP_45_","origin":"rep","source_instance":"0","source_type":"APP","time":"2016-11-22T16:29:37Z","timestamp":1479832177017862007}

Key value pairs are with in curly brackets { }
Example: cf_space_name":"RDS-DEV1" this is one key value pair

Can you please help us to write a regular expression for the above sample event.

0 Karma
1 Solution

sundareshr
Legend

Try this

REGEX = "([^"]+)":"([^"]+)
FORMAT = $1:$2

View solution in original post

0 Karma

sshelly_splunk
Splunk Employee
Splunk Employee

Try this one:

REGEX= (?ism)"(\w+)":"([^"]+)|timestamp":(?P\d+)
FORMAT = $1:$2

0 Karma

sundareshr
Legend

Try this

REGEX = "([^"]+)":"([^"]+)
FORMAT = $1:$2
0 Karma

ppablo
Retired

Apologies @sundareshr and @rajgowd1 I accidentally clicked the Accept button. I unaccepted it since this question is still unresolved.

0 Karma

rajgowd1
Communicator

Hi Sundaresh,
when i try index=myindex | rex "([^"]+)":"([^"]+)
getting error like Mismatched ']'. but the same expression is working in regex101

am i missing any here?

0 Karma

sundareshr
Legend

If you're using it in rex, you'll need to escape the quote. Try this

... | rex "\"(?<key>[^\"]+)\":\"(?<value>[^\"]+)" max_match=0

With this, key and value will be mvfields, to access a specific value, you will have to use mvindex(key, n) etc. I would not recommed this approach.

The other thing you could try is the extract command

... | extract kvdelim=":" pairdelim=","

This may work better for your case

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Extract

rajgowd1
Communicator

thank you.you are genius.regular expression and extract both are working as expected

Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...