topic Re: Multiple Key Value Pair Extraction in Splunk Search

Multiple Key Value Pair Extraction

FRoth — Tue, 29 Apr 2014 15:03:48 GMT

I have a log format that contains KEY/VALUE pairs in this format:

Feb 10 12:02:38 192.168.56.101 Feb 10 12:02:37 PROMETHEUS/192.168.56.101 THOR: Info: MODULE: UserAccounts MESSAGE: User Account USER: trinity FULL_NAME: trinity PRIV: 1 LAST_LOGON: 03/02/2014 13:52:38 BADPWCOUNT: 0 LOGON_SERVER: \\* NUM_LOGONS: 8 PASS_AGE: 78.00  days

The thing is, that the values are terminated by the next key, so the extraction would be:

(?<_KEY_1>[A-Z_]+): (?<_VAL_1>.*?) [A-Z_]+:

The problem I face is that it only extracts the first, third, fifth, seventh ... key/value pair as it skips the following key as part of the first extraction.
Is there a way to handle this?

Re: Multiple Key Value Pair Extraction

richgalloway — Tue, 29 Apr 2014 15:26:31 GMT

Are the keys always the same and in the same order?

Re: Multiple Key Value Pair Extraction

FRoth — Tue, 29 Apr 2014 15:33:04 GMT

No. There are more than 40 different formats. I use to extract them separately by specific extraction rules and would like to extract all by a single rule. i.e. what I normally do is "DOMAIN: (?.*?) [A-Z_]+:".

Re: Multiple Key Value Pair Extraction

richgalloway — Tue, 29 Apr 2014 15:52:20 GMT

According to RegExr, this should work:

(?<_KEY_1>[A-Z_]+): (?<_VAL_1>.*?) (?=[A-Z_]+:)

It matches the next key without including it in the result.

Re: Multiple Key Value Pair Extraction

FRoth — Tue, 29 Apr 2014 15:54:34 GMT

Awesome. This works. Thanks

Re: Multiple Key Value Pair Extraction

FRoth — Mon, 28 Sep 2020 16:31:26 GMT

I changed it a bit so that it also includes the last key/value pair (?<KEY_1>[A-Z]+): (?<VAL_1>.*?)(?=(\s[A-Z]+:)|$)