Splunk Search

Rex to parse key/value tags in splunk

braicu
New Member

Hello all,
Please help me with some regular expression.
This is the text:
{"Value": "arn:aws:cloudformation:us-west-2:248901117996:stack/RCM-CloudFrontS3Route53/a94d3010-317a-11e9-8cfb-0a6b0666b1b0", "Key": "aws:cloudformation:stack-id"}, {"Value": "RCM-CloudFrontS3Route53", "Key": "aws:cloudformation:stack-name"}, {"Value": "WebsiteBucket", "Key": "aws:cloudformation:logical-id"}

And i need regular expression to extract when match exactly the key , for example aws:cloudformation:stack-id , the Value between ""
I need the final result 3 columns Stack-Id , Stack-Name, Logical-Id with their values below.
Please be aware there are multiple key-value pairs this is just one example so i need the rex very good to match the work key and then extract the value

Tags (2)
0 Karma

maciep
Champion

I would likely do this at search time in props & transforms, but I'm not sure if that something you're familiar with?

props.conf

[your:sourcetype]
REPORT-my_aws_fields = my_aws_fields

transforms.conf

[my_aws_fields]
REGEX = "Value":\s*"([^"]+)",\s*"Key":\s*"aws:cloudformation:([^"]+)
FORMAT = $2::$1

You should be able plug the regex into regex101 to see what it's doing, but essentially it's capturing each of the Value/Key pairs in your data. For each, it captures the entire value to the first group and then just the part after aws:cloudformation: in the key to the second group. And then we tell splunk to create a field whose name is the second capture group and whose value is the first capture group.

So the end result should be those fields with those values. And more generically, any aws:cloudformation:* keys with their values.

0 Karma

niketn
Legend

@braicu your data seems to be JSON. Splunk should be able to do automatic Search Time Field Extraction (even INDEX TIME with right configuration, if you really want it), using KV_MODE=json

You can try the following run anywhere example based on your sample data where I have used spath (which can parse and extract KV pair from JSON or XML). However, you should try KV_MODE with your sample data first:

| makeresults 
| eval _raw="[{
    \"Value\": \"arn:aws:cloudformation:us-west-2:248901117996:stack/RCM-CloudFrontS3Route53/a94d3010-317a-11e9-8cfb-0a6b0666b1b0\",
    \"Key\": \"aws:cloudformation:stack-id\"
    }, {
    \"Value\": \"RCM-CloudFrontS3Route53\",
    \"Key\": \"aws:cloudformation:stack-name\"
    }, {
    \"Value\": \"WebsiteBucket\",
    \"Key\": \"aws:cloudformation:logical-id\"
    }]"
| spath
| fields - _*
| rename "{}.Key" as Key, "{}.Value" as Value
| eval data=mvzip(Key,Value,"###")
| fields data
| mvexpand data
| makemv data delim="###"
| eval Key=mvindex(data,0), Value=mvindex(data,1)
| fields Key Value
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

richgalloway
SplunkTrust
SplunkTrust

What have you tried so far?
Are you familiar with regex101.com? It's a great site for testing regular expressions.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...