Splunk Search

Extract key-value pairs

scottkoontz57
New Member

I'm trying to extract the key-value pairs from an Untangle firewall log ( syslog ), but the Regex example I found on the forum don't work and I'm not even sure that's the best approach.
Example:

May  9 16:19:33 192.168.10.254 May  9 16:19:34 INFO  uvm[0]:  {"timeStamp":"2019-05-09 16:19:34.148","s2pBytes":0,"p2sBytes":0,"endTime":1557443974148,"sessionId":102006119003906,"tag":"uvm[0]: ","class":"class com.untangle.uvm.app.SessionStatsEvent","sessionEvent":{"entitled":true,"partitionTablePostfix":"_2019_05_09","hostname":"192.168.10.102","CServerPort":161,"protocol":17,"protocolName":"UDP","tag":"uvm[0]: ","localAddr":"/192.168.10.102","class":"class com.untangle.uvm.app.SessionEvent","SServerAddr":"/192.168.0.192","remoteAddr":"/192.168.0.192","serverIntf":100,"CClientAddr":"/192.168.10.102","serverCountry":"XU","sessionId":102006119003906,"SClientAddr":"/97.115.239.26","clientCountry":"XL","CClientPort":65181,"policyRuleId":0,"timeStamp":"2019-05-09 16:19:34.148","clientIntf":2,"policyId":1,"SClientPort":65181,"bypassed":false,"SServerPort":161,"CServerAddr":"/192.168.0.192","tagsString":""},"c2pBytes":0,"p2cBytes":0,"partitionTablePostfix":"_2019_05_09"}
0 Karma

landen99
Motivator

intransforms.conf

[my_transform]
REGEX = "([^",{}]+)":"?([^",{}]+)"?
FORMAT = $1::$2
MV_ADD = true

add the props to call it: REPORT-mytransform=mytransform.

0 Karma

venkasplunk
New Member

Hello, use this. simple and powerful.

My Log format - |cb=hpot|et=2222|ip=x.x.x.x|action=acv|

extract pairdelim="{|}" kvdelim="=" | table cb,et,ip,action -----> Here pairdelim is | and kvdelim (keyvalue delim is 😃

For your case, seems like pairdelim is , and kvdelim is :

0 Karma

woodcock
Esteemed Legend

Try this:

REGEX = "([^"]*)":(?:(?:{[^{}]+)}|(?:"?[^,"]*)"?)(?=$|,)
FORMAT = $1::$2
MV_ADD = true

See it work here:
https://regex101.com/r/9Iyyy9/1

0 Karma

sumanssah
Communicator

If you are using "syslog" mechanism to send logs from untangle to Splunk, I would suggest using below mentioned SPL for field extraction

index="untangle" 
| rex "(?<json>\{.+)" | spath input=json | fields - json 
| replace /* with * in CClientAddr,CServerAddr,SClientAddr,SServerAddr,localAddr,remoteAddr
0 Karma

scottkoontz57
New Member

Can you elaborate? What exactly would I do with the code you provided?

0 Karma

sumanssah
Communicator

this will perform search time field extraction for untangle logs.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is the regex you're using?

---
If this reply helps you, Karma would be appreciated.
0 Karma

scottkoontz57
New Member
"([^"]+)":"([^"]+)
0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  &#x1f680; Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...