Splunk Search

Extract Field Name and Value from Data Source using Delimiter based KV extraction

jspigler2010
Explorer

I'm looking to dynamically extract both the field name and the associated value from a data source. Essentially, the field name is a session variable on F5 and these variables can be added and removed at will. So statically trying to regex the field extraction and field name would be impossible. Here is a data sample

hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.end",Session_Variable_Value="Nov 26 22:14:03 2020 GMT"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.exist",Session_Variable_Value="1"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.issuer",Session_Variable_Value="DC=local, DC=cloudmegalodon, CN=cloudmegalodon-CMDC1-CA"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.serial",Session_Variable_Value="59:00:00:00:09:21:54:f7:30:0b:fd:f7:9f:00:00:00:00:00:09"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.start",Session_Variable_Value="Nov 27 22:14:03 2018 GMT"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.subject",Session_Variable_Value="DC=local, DC=cloudmegalodon, OU=OrgUsers, CN=TINA F. OLSON.1468013579"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.valid",Session_Variable_Value="0"

I want to take for example Session_Variable_Name="session.ssl.cert.serial",Session_Variable_Value="59:00:00:00:09:21:54:f7:30:0b:fd:f7:9f:00:00:00:00:00:09"
and make session.ssl.cert.serial the field name and 59:00:00:00:09:21:54:f7:30:0b:fd:f7:9f:00:00:00:00:00:09 the field value.

I have not been able to get this to work with kv pairdelim and kvdelim. Maybe there is a better way to do this. Open to any and all ideas!

Here is the search I'm working with now

index="test_f5" sourcetype="f5_syslog_splunk" | kv pairdelim="Session_Variable_Name\r\n" kvdelim="\",Session_Variable_Value=\""
0 Karma
1 Solution

harsmarvania57
Ultra Champion

Hi,

You can achieve this using regular expression in props and transforms to extract field and value at search time.

On Search Head
props.conf

[yourSourceType]
REPORT-test = extract_session_kv

transforms.conf

[extract_session_kv]
CLEAN_KEYS = 0
FORMAT = $1::$2
REGEX = Session_Variable_Name\=\"([^"]+)\"\,Session_Variable_Value\=\"([^"]+)\"

View solution in original post

0 Karma

harsmarvania57
Ultra Champion

Hi,

You can achieve this using regular expression in props and transforms to extract field and value at search time.

On Search Head
props.conf

[yourSourceType]
REPORT-test = extract_session_kv

transforms.conf

[extract_session_kv]
CLEAN_KEYS = 0
FORMAT = $1::$2
REGEX = Session_Variable_Name\=\"([^"]+)\"\,Session_Variable_Value\=\"([^"]+)\"
0 Karma

jspigler2010
Explorer

Works great! Thanks harsmarvania!

0 Karma
Get Updates on the Splunk Community!

ATTENTION!! We’re MOVING (not really)

Hey, all! In an effort to keep this Slack workspace secure and also to make our new members' experience easy, ...

Splunk Admins: Build a Smarter Stack with These Must-See .conf25 Sessions

  Whether you're running a complex Splunk deployment or just getting your bearings as a new admin, .conf25 ...

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...