Splunk Search

How to create a regex to capture information from an indexing sourcetype?

lukeandrews
New Member

Hi,

I'm struggling to create a regex to capture all the information correctly from a sourcetype we have and make them into interesting fields.

The structure of the logs is:

username: "User1"; companyName: "Company 4"; etc etc

Where there's no information within the field it remains empty "".

When trying it doesn't seem to pick up all results 😞

Please help.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this.

username: "(?<username>\w*)";\s+companyName: "(?<companyName>[\w ]*)";
---
If this reply helps you, Karma would be appreciated.
0 Karma

sundareshr
Legend

Try these changes to props & transforms
*UPDATED*
props.conf

REPORT-extractfields

transforms.conf

[extractfields]
REGEX = (\w+):\s+\"([^"]+)?\"
FORMAT = $1::$2

Restart Splunk.

lukeandrews
New Member

I tried but it didn't work 😞 Should this automatically add values to my interesting fields?

0 Karma

sundareshr
Legend

It will show in interesting fields only if more than 20% of the events have the field. Try the updated regex. Also, the props & transforms need to be on your search head(s)

0 Karma

cmerriman
Super Champion

can you share an example of the log or the regex you've tried?
also, regex101.com is an excellent place to try to work it out.

0 Karma

lukeandrews
New Member

action: "DOC_UPLOAD_DOCUMENTS"; username: "zCarrier4User1"; companyName: "zCarrier 4"; consolidationId: "19882"; shipmentReference: "krish-9165"; vessel: "zCarrier4Vessel1"; vesselImo: "1000004"; taskId: "56838"; consignmentId: "29726"; parcelReference: "45-p-3"; consignor: "zTerminal"; consignee: ""; billOfLadingDate: "8 AUG 2016"; cargo: "Barley edited"; quantity: "121,212 Pounds"; loadingLocation: "zWestwego"; destination: "Visakhapatnam INDIA"; freightForwarder: ""; presentee: ""; financeNumber: ""; endorseeCompanyName: "zTerminal"; endorsementStampType: ""; endorsementType: ""; jSessionId: "0F78869A26215A2AC3800D96DE92CDCC";

This is an example of what I need to extract.

0 Karma

Honey0308
Explorer

Hi Lukeandrews,

Even I am new to Splunk, but I suggest you can make use of rex command to extract the required information in the Search String.
We notice semi-colon marking end of a key-value pair, so we can use something like this :

host=ABC index =XYZ sourcetype=PQR |rex field=_raw "username:\"(?<username>\w*)\";" | rex field=_raw "companyName:\"(?<Company>[\w\s]*)\";" .....so on

If the values ("User1"), are not enclosed in quotations, same can be used as:

host=ABC index =XYZ sourcetype=PQR |rex field=_raw "username:(?<username>\w*);" | rex field=_raw "companyName:(?<Company>[\w\s]*);" .... so on

Hope this helps...:)

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...