Regex in props.conf doesn't work

[nmohammed]

Our application logs events to the Windows application events with custom SourceNames. Need help to extract the fields using the props and transforms. I am able to extract the fields search time using the rex command, but the same doesn't work in props.conf .

tried extracting one field -

REGEX working in search - rex field=Message "^[^\]\n]*\]\s+(?P\w+)"

props.conf

[ebs_prod_api]
REPORT-ebs_type = ebs_type_extract

transforms.conf

[ebs_type_extract]
SOURCE_KEY = Message
REGEX = ^[^\]\n]*\]\s+(?P\w+)

Need to extract the following fields -
Log_type - VERBOSE , ThreadId - {117}
CorrelationId, Http status Code, Duration, Execution, Session ID .

Sample data -

07/03/2019 11:33:38 AM
LogName=Application
SourceName=exs_nmon
EventCode=0
EventType=4
Type=Information
ComputerName=ex1nmkilo
TaskCategory=%1
OpCode=Info
RecordNumber=19214030
Keywords=Classic
Message=[07/03/19 11:33:38.9356] **VERBOSE** **{117}** (NewFolder): (**CorrelationId**: IO-78904a2a-f22b-43bc-b39c-1188e9530622) Read Data for file version "Pipeline folder 2030\{1d38cda6-13f7-4353-bd46-8bde2659af97}" from \\nmsxlfs\jkil11638\nuimo1927\encdata\files\Pipeline folder 2030\{1d38cda6-13f7-4353-bd46-8bde2659af97}\Versions\00135_file.mr

07/03/2019 01:26:54 PM
LogName=Application
SourceName=exs_nmon
EventCode=0
EventType=4
Type=Information
ComputerName=ex1nmkilo
TaskCategory=%1
OpCode=Info
RecordNumber=32205553
Keywords=Classic
Message=[07/03/19 13:26:54.8825] RESTAPI {121} (GET-/v1/folder/{fileid}/metadata): Instance ID: xcert123987; User ID: one_lo; **CorrelationId**: io19db84-f68b-4328-93eb-963cf77f0feb; **Execution**: Complete; **Duration**: 260 ms; **Http status Code**: 200; **Session ID**: ui90k2b4-b714-4caa-ab26-7d0ee7df3681; Node: enc169094num; **PODVersion**: 13.1; fileid: 29006663-92a5-4ff9-ad3b-64f387004cf7

Note - Not all events contain all the fields.

Thanks
Naveed

[woodcock]

Use this setting in transforms.conf:

[ebs_type_extract]
SOURCE_KEY = Message
REGEX = ^[^]\n]*]\s+(\w+)
FORMAT = <YourNewFieldNameHere>::$1

You need to deploy this to your Search Head, set your search mode to Verbose and use this to start:

index=* sourcetype=ebs_prod_api | table <YourNewFieldNameHere>

[cpetterborg]

Can you fix your information above by using the 101010 button to keep Answers from interpreting things like <fieldname> improperly? That will make I much easier to answer you question.