Custom parsing

[yhemaraj]

I am rookie here.
I have a log of type
"2e 00000008 M 2050 nodemgr 09/10/21 20:01:11.860361 NODEMGR: Successfully set our time"
I would like to extract the fields as below.
deviceId moduleId level id moduleName time(YY,mm,DD HH:MM:SS) message
I do not want to parse the message at this point, but may want to parse a subset of structured messages at a later point.
How do I go about doing this?

[emiller42]

So two things you'll want to do:

First, create a stanza in transforms.conf that uses regex to parse out your fields. (Below is an example, which is based on the one line you posted. It may need to be tweaked)

[sourcetype_extraction]
REGEX = (\w+)\s+(\d+)\s+(\w+)\s+(\d+)\s+(\w+)\s+(\d+/\d+/\d+\s\d+:\d+:\d+\.\d+)\s(.*)
FORMAT = deviceId::$1 moduleId::$2 level::$3 id::$4 moduleName::$5 time::$6 message::$7

Then you'll want to create a stanza in your props.conf that applies the transform to your sourcetype.

[sourcetype]
REPORT-sourcetype = sourcetype_extraction

All of this is applied at search time, so will apply to anything you've already indexed, and can be changed without losing anything.

It's also worth noting that the timestamp should be getting extracted on index into the _time field, so you shouldn't have to explicitly pull it out. But it may be a good idea to do so anyway via the TIMEFORMAT setting in props.conf.