Splunk Search

Rex to parse key/value tags in splunk

braicu
New Member

Hello all,
Please help me with some regular expression.
This is the text:
{"Value": "arn:aws:cloudformation:us-west-2:248901117996:stack/RCM-CloudFrontS3Route53/a94d3010-317a-11e9-8cfb-0a6b0666b1b0", "Key": "aws:cloudformation:stack-id"}, {"Value": "RCM-CloudFrontS3Route53", "Key": "aws:cloudformation:stack-name"}, {"Value": "WebsiteBucket", "Key": "aws:cloudformation:logical-id"}

And i need regular expression to extract when match exactly the key , for example aws:cloudformation:stack-id , the Value between ""
I need the final result 3 columns Stack-Id , Stack-Name, Logical-Id with their values below.
Please be aware there are multiple key-value pairs this is just one example so i need the rex very good to match the work key and then extract the value

Tags (2)
0 Karma

maciep
Champion

I would likely do this at search time in props & transforms, but I'm not sure if that something you're familiar with?

props.conf

[your:sourcetype]
REPORT-my_aws_fields = my_aws_fields

transforms.conf

[my_aws_fields]
REGEX = "Value":\s*"([^"]+)",\s*"Key":\s*"aws:cloudformation:([^"]+)
FORMAT = $2::$1

You should be able plug the regex into regex101 to see what it's doing, but essentially it's capturing each of the Value/Key pairs in your data. For each, it captures the entire value to the first group and then just the part after aws:cloudformation: in the key to the second group. And then we tell splunk to create a field whose name is the second capture group and whose value is the first capture group.

So the end result should be those fields with those values. And more generically, any aws:cloudformation:* keys with their values.

0 Karma

niketn
Legend

@braicu your data seems to be JSON. Splunk should be able to do automatic Search Time Field Extraction (even INDEX TIME with right configuration, if you really want it), using KV_MODE=json

You can try the following run anywhere example based on your sample data where I have used spath (which can parse and extract KV pair from JSON or XML). However, you should try KV_MODE with your sample data first:

| makeresults 
| eval _raw="[{
    \"Value\": \"arn:aws:cloudformation:us-west-2:248901117996:stack/RCM-CloudFrontS3Route53/a94d3010-317a-11e9-8cfb-0a6b0666b1b0\",
    \"Key\": \"aws:cloudformation:stack-id\"
    }, {
    \"Value\": \"RCM-CloudFrontS3Route53\",
    \"Key\": \"aws:cloudformation:stack-name\"
    }, {
    \"Value\": \"WebsiteBucket\",
    \"Key\": \"aws:cloudformation:logical-id\"
    }]"
| spath
| fields - _*
| rename "{}.Key" as Key, "{}.Value" as Value
| eval data=mvzip(Key,Value,"###")
| fields data
| mvexpand data
| makemv data delim="###"
| eval Key=mvindex(data,0), Value=mvindex(data,1)
| fields Key Value
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

richgalloway
SplunkTrust
SplunkTrust

What have you tried so far?
Are you familiar with regex101.com? It's a great site for testing regular expressions.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Fall Into Learning with New Splunk Education Courses

Every month, Splunk Education releases new courses to help you branch out, strengthen your data science roots, ...

Super Optimize your Splunk Stats Searches: Unlocking the Power of tstats, TERM, and ...

By Martin Hettervik, Senior Consultant and Team Leader at Accelerate at Iver, Splunk MVPThe stats command is ...

How Splunk Observability Cloud Prevented a Major Payment Crisis in Minutes

Your bank's payment processing system is humming along during a busy afternoon, handling millions in hourly ...