How do I use props.conf and or transforms.conf to parse log file?

[eholz1]

I would like to use props.conf and/or transforms.conf to parse data coming from a generic

single line log file using regex to search for "Error" or "Notice" I did a test for my regex in regex 101, and the regex seems ok. regex = (?<=Error)(.*$)

I do have a sourcetype for the incoming data - what should I be looking for and what files should I edit to

allow this?

Thanks,

eholz1

[richgalloway]

To parse that one field at search-time (the best option, IMO), you can use an EXTRACT setting in props.conf.  No transform is needed.

[mysourcetype]
EXTRACT-error = Error(?<Error>.*)$

Note the required named capture group in the regex.  Also, avoid using lookbehind in regexes as they're expensive and usually unnecessary.

[eholz1]

hello richgallowy,

I will give this a shot, and see what happens.

and come back if I have any questions.  could I add an "OR" to this like: 

EXTRACT-error = Error(?<Error>.*)$

OR Notice(?<Notice>.*)$

thanks for the support,

eholz1

[richgalloway · Accepted solution]

There is nothing to stop you from doing that, but it may not do what you expect.

If you need to extract both Error and Notice fields then use two EXTRACT settings.

EXTRACT-error = Error(?<Error>.*)$
EXTRACT-notice = Notice(?<Notice>.*)$

[eholz1]

Thanks again for the tips.

Now one last dump question - how do I verify that this works?

Do I need any other settings in the props.conf file to allow the data to be viewed in a search?

When I configure the props.conf file with the source "my_source", I still see all the data in the log file that i do not need.

Thanks again,

eholz1

[richgalloway]

You'll know the extractions work when you see the new fields on the left side of the results or when you see values from | table Error Notice.

Yes, you will still see field "you do not need".  Nothing in the OP asked about getting rid of anything.  You probably don't want to get rid of anything at this points, because you may need it for another use case.  You can, however, only show what you need by using the table command as above.

[eholz1]

Thanks again for the clarification - I am still learning!