Splunk Search

Regex in props.conf doesn't work

Contributor

Our application logs events to the Windows application events with custom SourceNames. Need help to extract the fields using the props and transforms. I am able to extract the fields search time using the rex command, but the same doesn't work in props.conf .

tried extracting one field -

REGEX working in search - rex field=Message "^[^\]\n]*\]\s+(?P\w+)"

props.conf

[ebs_prod_api]
REPORT-ebs_type = ebs_type_extract

transforms.conf

[ebs_type_extract]
SOURCE_KEY = Message
REGEX = ^[^\]\n]*\]\s+(?P\w+)

Need to extract the following fields -
Log_type - VERBOSE , ThreadId - {117}
CorrelationId, Http status Code, Duration, Execution, Session ID .

Sample data -

07/03/2019 11:33:38 AM
LogName=Application
SourceName=exs_nmon
EventCode=0
EventType=4
Type=Information
ComputerName=ex1nmkilo
TaskCategory=%1
OpCode=Info
RecordNumber=19214030
Keywords=Classic
Message=[07/03/19 11:33:38.9356] **VERBOSE** **{117}** (NewFolder): (**CorrelationId**: IO-78904a2a-f22b-43bc-b39c-1188e9530622) Read Data for file version "Pipeline folder 2030\{1d38cda6-13f7-4353-bd46-8bde2659af97}" from \\nmsxlfs\jkil11638\nuimo1927\encdata\files\Pipeline folder 2030\{1d38cda6-13f7-4353-bd46-8bde2659af97}\Versions\00135_file.mr

07/03/2019 01:26:54 PM
LogName=Application
SourceName=exs_nmon
EventCode=0
EventType=4
Type=Information
ComputerName=ex1nmkilo
TaskCategory=%1
OpCode=Info
RecordNumber=32205553
Keywords=Classic
Message=[07/03/19 13:26:54.8825] RESTAPI {121} (GET-/v1/folder/{fileid}/metadata): Instance ID: xcert123987; User ID: one_lo; **CorrelationId**: io19db84-f68b-4328-93eb-963cf77f0feb; **Execution**: Complete; **Duration**: 260 ms; **Http status Code**: 200; **Session ID**: ui90k2b4-b714-4caa-ab26-7d0ee7df3681; Node: enc169094num; **PODVersion**: 13.1; fileid: 29006663-92a5-4ff9-ad3b-64f387004cf7

Note - Not all events contain all the fields.

Thanks
Naveed

0 Karma

Esteemed Legend

Use this setting in transforms.conf:

[ebs_type_extract]
SOURCE_KEY = Message
REGEX = ^[^]\n]*]\s+(\w+)
FORMAT = <YourNewFieldNameHere>::$1

You need to deploy this to your Search Head, set your search mode to Verbose and use this to start:

index=* sourcetype=ebs_prod_api | table <YourNewFieldNameHere>
0 Karma

SplunkTrust
SplunkTrust

Can you fix your information above by using the 101010 button to keep Answers from interpreting things like <fieldname> improperly? That will make I much easier to answer you question.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!