Splunk Search

Extract Field Name and Value from Data Source using Delimiter based KV extraction

jspigler2010
Explorer

I'm looking to dynamically extract both the field name and the associated value from a data source. Essentially, the field name is a session variable on F5 and these variables can be added and removed at will. So statically trying to regex the field extraction and field name would be impossible. Here is a data sample

hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.end",Session_Variable_Value="Nov 26 22:14:03 2020 GMT"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.exist",Session_Variable_Value="1"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.issuer",Session_Variable_Value="DC=local, DC=cloudmegalodon, CN=cloudmegalodon-CMDC1-CA"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.serial",Session_Variable_Value="59:00:00:00:09:21:54:f7:30:0b:fd:f7:9f:00:00:00:00:00:09"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.start",Session_Variable_Value="Nov 27 22:14:03 2018 GMT"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.subject",Session_Variable_Value="DC=local, DC=cloudmegalodon, OU=OrgUsers, CN=TINA F. OLSON.1468013579"
hostname="ip-10-1-1-49.us-gov-west-1.compute.internal",errdefs_msgno="01490007:6:",partition_name="Common",session_id="c3da4e31",Access_Profile="/Common/testPolicy1",Partition="Common",Session_Id="c3da4e31",Session_Variable_Name="session.ssl.cert.valid",Session_Variable_Value="0"

I want to take for example Session_Variable_Name="session.ssl.cert.serial",Session_Variable_Value="59:00:00:00:09:21:54:f7:30:0b:fd:f7:9f:00:00:00:00:00:09"
and make session.ssl.cert.serial the field name and 59:00:00:00:09:21:54:f7:30:0b:fd:f7:9f:00:00:00:00:00:09 the field value.

I have not been able to get this to work with kv pairdelim and kvdelim. Maybe there is a better way to do this. Open to any and all ideas!

Here is the search I'm working with now

index="test_f5" sourcetype="f5_syslog_splunk" | kv pairdelim="Session_Variable_Name\r\n" kvdelim="\",Session_Variable_Value=\""
0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

You can achieve this using regular expression in props and transforms to extract field and value at search time.

On Search Head
props.conf

[yourSourceType]
REPORT-test = extract_session_kv

transforms.conf

[extract_session_kv]
CLEAN_KEYS = 0
FORMAT = $1::$2
REGEX = Session_Variable_Name\=\"([^"]+)\"\,Session_Variable_Value\=\"([^"]+)\"

View solution in original post

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

You can achieve this using regular expression in props and transforms to extract field and value at search time.

On Search Head
props.conf

[yourSourceType]
REPORT-test = extract_session_kv

transforms.conf

[extract_session_kv]
CLEAN_KEYS = 0
FORMAT = $1::$2
REGEX = Session_Variable_Name\=\"([^"]+)\"\,Session_Variable_Value\=\"([^"]+)\"

View solution in original post

0 Karma

jspigler2010
Explorer

Works great! Thanks harsmarvania!

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!