topic Re: Key-value pair extraction in Security

Key-value pair extraction

jkloet — Wed, 23 Nov 2011 16:42:45 GMT

Given the event (output from Cisco ASA 'show vpn-sessiondb full svc' command)...

Session ID: 33397 | EasyVPN: 0 | Username: user@company.com | Group: VPN-User | Tunnel Group: WebVPN-AD-Authentication | IP Addr: 192.168.1.1 | Public IP: 184.151.1.1 | Protocol: Clientless SSL-Tunnel DTLS-Tunnel | License: SSL VPN | Session Subtype: With client | Encryption: RC4 AES128 | Login Time: 15:35:44 EST Tue Nov 22 2011 | Duration: 0h:00m:35s | Inactivity: 0h:00m:00s | Bytes Tx: 20277 | Bytes Rx: 19574 | NAC Result: Unknown | Posture Token: | VLAN Mapping: N/A | VLAN: 0 ||

... I am able to extract the key-value pairs using:

**props.conf
[vpnsessiondata]

DATETIME_CONFIG=CURRENT

REPORT-sessiondata=sessiondata

**transforms.conf

[sessiondata]

DELIMS = "|", ":"

The problem during searches is that a field is ignored if the character delimiting the key from the value (":" in this case) is also contained within the value (any time value for instance).

How can I deal with this?

Thanks! Jeff

Re: Key-value pair extraction

tpsplunk — Wed, 23 Nov 2011 17:30:18 GMT

I don't think you can use DELIMS to do this because as you've noted it doesn't work well when one of your delimiters shows up in a key or value field. I had to solve this by using a REGEX instead, see: http://splunk-base.splunk.com/answers/34550/help-with-regex-to-separate-keyvalue-pairs-with-a-character-sequence

in your case you would want to craft a regex that captures into two capture groups, group 1 is the key and group 2 is the value.

REGEX = To be created
FORMAT = $1::$2

i will try to figure out the regex, but my regex-fu is of a medium level so hopefully someone will beat me to it.

Re: Key-value pair extraction

_d_ — Wed, 23 Nov 2011 17:35:08 GMT

Not sure whether this will work, but give it a try.

transforms.conf [sessiondata] DELIMS = "|", ": "

Notice the space after : in DELIMS

Hope this helps.

> please upvote and accept answer if you find it useful - thanks!

Re: Key-value pair extraction

jkloet — Wed, 23 Nov 2011 17:47:14 GMT

That would be nice however the documentation says:

[multiple_delims]
DELIMS = "|;", "=:"

*The above example extracts key-value pairs which are separated by '|' or ';'.
*while the key is delimited from value by '=' or ':'.

Re: Key-value pair extraction

_d_ — Wed, 23 Nov 2011 18:04:02 GMT

OK, in that case then try something like this using your transforms stanza:

transforms.conf [sessiondata] REGEX = ([^\|]+):\s([^\|]+) FORMAT = $1::$2

REGEX reads "anything that is not a pipe, followed by a colon, followed by a space, followed by a pipe, followed by anything that is not a pipe.

Hope this helps.

> please upvote and accept answer if you find it useful - thanks!

Re: Key-value pair extraction

jkloet — Wed, 23 Nov 2011 18:40:09 GMT

That works as needed... thanks!! Jeff