How to create a regex to capture information from an indexing sourcetype?

[lukeandrews]

Hi,

I'm struggling to create a regex to capture all the information correctly from a sourcetype we have and make them into interesting fields.

The structure of the logs is:

username: "User1"; companyName: "Company 4"; etc etc

Where there's no information within the field it remains empty "".

When trying it doesn't seem to pick up all results 😞

Please help.

[richgalloway]

Try this.

username: "(?<username>\w*)";\s+companyName: "(?<companyName>[\w ]*)";

[sundareshr]

Try these changes to props & transforms
*UPDATED*
props.conf

REPORT-extractfields

transforms.conf

[extractfields]
REGEX = (\w+):\s+\"([^"]+)?\"
FORMAT = $1::$2

Restart Splunk.

[lukeandrews]

I tried but it didn't work 😞 Should this automatically add values to my interesting fields?

[sundareshr]

It will show in interesting fields only if more than 20% of the events have the field. Try the updated regex. Also, the props & transforms need to be on your search head(s)

[cmerriman]

can you share an example of the log or the regex you've tried?
also, regex101.com is an excellent place to try to work it out.

[lukeandrews]

action: "DOC_UPLOAD_DOCUMENTS"; username: "zCarrier4User1"; companyName: "zCarrier 4"; consolidationId: "19882"; shipmentReference: "krish-9165"; vessel: "zCarrier4Vessel1"; vesselImo: "1000004"; taskId: "56838"; consignmentId: "29726"; parcelReference: "45-p-3"; consignor: "zTerminal"; consignee: ""; billOfLadingDate: "8 AUG 2016"; cargo: "Barley edited"; quantity: "121,212 Pounds"; loadingLocation: "zWestwego"; destination: "Visakhapatnam INDIA"; freightForwarder: ""; presentee: ""; financeNumber: ""; endorseeCompanyName: "zTerminal"; endorsementStampType: ""; endorsementType: ""; jSessionId: "0F78869A26215A2AC3800D96DE92CDCC";

This is an example of what I need to extract.

[Honey0308]

Hi Lukeandrews,

Even I am new to Splunk, but I suggest you can make use of rex command to extract the required information in the Search String.
We notice semi-colon marking end of a key-value pair, so we can use something like this :

host=ABC index =XYZ sourcetype=PQR |rex field=_raw "username:\"(?<username>\w*)\";" | rex field=_raw "companyName:\"(?<Company>[\w\s]*)\";" .....so on

If the values ("User1"), are not enclosed in quotations, same can be used as:

host=ABC index =XYZ sourcetype=PQR |rex field=_raw "username:(?<username>\w*);" | rex field=_raw "companyName:(?<Company>[\w\s]*);" .... so on

Hope this helps...:)