Splunk Search

How to extract curly brackets in regular expression

ssaenger
Communicator

Hi,

I am having a problem extracting fields that have curly brackets {}
I have the log file line;
2015.06.24 11:55:13.567;:;12.34.567.241;:;somehost;:;21947UIGHFKD99HKW8R;:;deviceId;:;F90HSDUC0A49A2C;:;1001;:;Ref=0003313C;:;303;:;320;:;28;:;0xA5;:;co.cab.infra.exception.ApplicationException: tlv ids of DeviceId=DeviceId{DeviceId='27896RAWG96B'} is null or empty; applicationInstanceId=APPLICATION_INSTANCE_3;:;siteId;:;siteUid;:;

and i am using the reg ex to extract the fields;

(?<date_time>\d{4}\.\d{2}\.\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3});:;(?<ip>\d+\.\d+\.\d+\.\d+);:;(?<host>[a-z0-9]+);:;(?<requester_id>[0-9A-F]+);:;(?<req_id>.+);:;(?<drm_domain_id>.+);:;(?<status>.+);:;(?<data>.+);:;(?<request_type>.+);:;(?<device_id>.+);:;(?<latency>.+);:;(?<col12>.+);:;(?<col13>.+);:;(?<site_id>.+);:;(?<site_uid>.+);:;

(Please note that i know the >< are incorrect but its the only way i could get it display on the forum! I am that new to this :))

however Splunk is not extracting col13, for this line. Other lines are extracted fine and i believe its due to the Curly Bracket. i have tried to extract the line by delimiting it /{ and entering all the other characters, however this has not worked.
Obviously the best way would have been to transform it however Splunk only supports one delimiting value and mine is ;:;

Thanks in advance.

Tags (2)
0 Karma
1 Solution

woodcock
Esteemed Legend

Try this:

(?<date_time>\d{4}\.\d{2}\.\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3});:;(?<ip>\d+\.\d+\.\d+\.\d+);:;(?<host>[^;]+);:;(?<requester_id>[^;]+);:;(?<req_id>[^;]+);:;(?<drm_domain_id>[^;]+);:;(?<status>[^;]+);:;(?<data>[^;]+);:;(?<request_type>[^;]+);:;(?<device_id>[^;]+);:;(?<latency>[^;]+);:;(?<col12>[^;]+);:;(?<col13>.+?);:;(?<site_id>[^;]+);:;(?<site_uid>[^;]+);:;

The problem is both greediness and some of your character classes (I use [^;] everywhere instead).

View solution in original post

0 Karma

woodcock
Esteemed Legend

Try this:

(?<date_time>\d{4}\.\d{2}\.\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3});:;(?<ip>\d+\.\d+\.\d+\.\d+);:;(?<host>[^;]+);:;(?<requester_id>[^;]+);:;(?<req_id>[^;]+);:;(?<drm_domain_id>[^;]+);:;(?<status>[^;]+);:;(?<data>[^;]+);:;(?<request_type>[^;]+);:;(?<device_id>[^;]+);:;(?<latency>[^;]+);:;(?<col12>[^;]+);:;(?<col13>.+?);:;(?<site_id>[^;]+);:;(?<site_uid>[^;]+);:;

The problem is both greediness and some of your character classes (I use [^;] everywhere instead).

0 Karma

ssaenger
Communicator

Thanks Woodcock, that worked. 🙂

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You could always use a look-behind.. If your trying to extract the Device ID you can do

(?<Extracted_Field>\{DeviceId\=\'[A-Z0-9]{12}\'\}(?=\s is\null\sor))
0 Karma

richgalloway
SplunkTrust
SplunkTrust

According to www.regex101.com, your regex string is failing in the requester_id field. This regex works.

(?<date_time>\d{4}\.\d{2}\.\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3});:;(?<ip>\d+\.\d+\.\d+\.\d+);:;(?<host>[a-z0-9]+);:;(?<requester_id>[0-9A-Z]+);:;(?<req_id>.+);:;(?<drm_domain_id>.+);:;(?<status>.+);:;(?<data>.+);:;(?<request_type>.+);:;(?<device_id>.+);:;(?<latency>.+);:;(?<col12>.+);:;(?<col13>.+);:;(?<site_id>.+);:;(?<site_uid>.+);:;
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...