All Apps and Add-ons

How to extract fields from the message of a windows event log?

Explorer

Hi all,

I have a problem extracting fields after the Message= portion of a windows application event log. Everything after Message= is being added as single value to the message field.

I think it may be the due to the formatting of the data after Message=, which seems to have a leading space or tab. It looks like this:

Message=Process started in passive mode.

Command Line: C:\something\something\something.exe
Process Id: 8988
...

(the the white space is being removed from this post - its in front of 'Command' and 'Process')

There are about 50 more fields after these, all on new lines but with a space or tab before them.

I have tried playing with the regex from transforms.conf below, but I've only been able to move the problem from the message field to a new field called Command_Line that contains a value of everything below it.

[wel-col-kv]
SOURCEKEY = Message
REGEX = \n([^:\n\r]+):[ \t]++([^\n]*)
FORMAT = $1::$2
MV
ADD = true

I have found some similar questions asked but haven't been able to come up with a solution yet. Any help would be greatly appreciated.

0 Karma
1 Solution

Explorer

I do have the TA, although I don't believe it's actually needed as windows event extraction is included in the default transforms.conf

I managed to get it working with the following changes to the default:

REGEX = \n? \t:[ \t]++([^\r]*)

FYI this was to extract avecto defendpoint events

View solution in original post

0 Karma

Explorer

I do have the TA, although I don't believe it's actually needed as windows event extraction is included in the default transforms.conf

I managed to get it working with the following changes to the default:

REGEX = \n? \t:[ \t]++([^\r]*)

FYI this was to extract avecto defendpoint events

View solution in original post

0 Karma

Splunk Employee
Splunk Employee

Hi John, I have the same problem parsing the avecto logs from Message. I tried your new regex but it didn't solve it for me. Strangely though the default regex parses everything fine for me in regex101. Did you make any other changes at all other than to the wel-col-kv settings?

Thanks

0 Karma

Explorer

Hi jwindley,

this is what I have in my avecto app:

props.conf
[source::WinEventLog:Application]
REPORT-MESSAGE = avec-wel-message, avec-wel-eq-kv, avec-wel-col-kv
KV_MODE=none

Note the below settings are effectively legacy, in place here to handle

data coming from much much older forwarders (3.x & 4.x)

SHOULDLINEMERGE = false
MAX
TIMESTAMPLOOKAHEAD=30
LINE
BREAKER = (\r\n)
TRANSFORMS-FIELDS = strip-winevt-linebreaker

transforms.conf
[avec-wel-message]
REGEX = (?sm)^(?<premsg>.+)\nMessage=(?.+)$
CLEAN_KEYS = false

[avec-wel-eq-kv]
SOURCEKEY = _premsg
DELIMS = "\n","="
MV_ADD = true

[avec-wel-col-kv]
SOURCEKEY = Message
REGEX = \n? \t:[ \t]++([^\r]*)
FORMAT = $1::$2
MV
ADD = true

0 Karma

SplunkTrust
SplunkTrust

are you using the Windows TA?
https://splunkbase.splunk.com/app/742/

0 Karma