Splunk Search

Regex for items in single quotes

aramakrishnan
New Member

I have a bunch of logs which look like the following.

2017-05-01 18:36:16,885 UTC - DEBUG - testpod_service.segment_processor - [BTESTC4277:37223] returning requests="[{'status': 'request_automat_eventab', 'priority': -2868.565, 'confidence': 1.0, 'db': {'dur_sec': 2868.565, 'n_hours': 2}, 'onset': 66377515, 'event_name': 'CYCLE', 'offset': 66951227

For every segment that looks like 'field':'value'I want Splunk to treat the value in quotes BEFORE the : as the field name and the value in quotes AFTER the : as the value for the field (similar treatment to field="value", which is Splunk's default).

How do I achieve this through a regex treatment? For example, in the example above, I'd like my field and value pairs to be as follows:

status = request_automat_eventab
priority = -2868.565
confidence = 1.0
dur_sec=2868.565
n_hours=2

...and so on

Thanks in advance!

0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

You can use the _KEY_1 and _VAL_1 field names in your regex capture groups to get all of the key value pairs dynamically. For example, in your props.conf, under the sourcetype for these events:

[yoursourcetype]
EXTRACT-keysandvalues = \"\[{\'(?<_KEY_1>\S+)\': \'(?<_VAL_1>\S+)\'

This will take every value in single quotes before the colon as the field name and the value in single quotes after the colon as the value for that field. This will work for any format you can match in the extractions regex. As long as the key value pairs are consistent, it will grab them all.

Hope this helps.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Are you asking about at index time or at search time?

0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

If the field / value format is consistent, you can use <_KEY_1> and <_VAL_1> as your field names in your capture groups to dynamically extract all field value pairs. In your props.conf, under the sourcetype for these events add a field extraction like the following:

[yoursourcetypehere]
EXTRACT-keysandvals = \"\[{\'(?<_KEY_1>\S+)\': \'(?<_VAL_1>\S+)\'

You may have to play with the regex a bit, but this should work. It will take everything in single quotes before the colon as a field name and everything after the colon in single quotes as the value for that field. You can use this any time you have a consistent format for the pairs.

I had a customer example where the field was in brackets followed by the value:

[fieldname1]fieldvalue1[fieldname2]fieldvalue2

Hope this helps.

0 Karma

adonio
Ultra Champion

use the field extractor from GUIi? click an event -> actions -> extract fields -> new tab pops -> follow the prompts -> save -> search your fields
read more here:
http://docs.splunk.com/Documentation/Splunk/6.5.3/Knowledge/ExtractfieldsinteractivelywithIFX

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...