Splunk Search

Field Extraction in Message field of Windows Event Log

Path Finder


I have an application sending logs to the windows event log with a lognamename of ErrorLogs. The error log looks like this:

05/30/2012 03:17:42 PM











Message=ApplicationName: EventLogTestWriter

Severity: Informational

Error: Added newline before ApplicationName

StackTrace: N/A

All fields extract well in the autoextraction in search except for the first field 'ApplicationName:' Any ideas? I tried a new line break and while it does send ApplicationName to a new line I still don't get the field to extract.

Ultra Champion

All WinEvent logs go through the parsing rules defined in $SPLUNK_HOME/etc/system/local/props.conf and $SPLUNK_HOME/etc/system/local/transforms.conf.

These state the following;


LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv

etc, etc


REGEX = (?sm)^(?<_pre_msg>.+)\nMessage=(?<Message>.+)$
CLEAN_KEYS = false

SOURCE_KEY = _pre_msg
DELIMS     = "\n","="
MV_ADD     = true

SOURCE_KEY = Message
REGEX      = \n([^:\n\r]+):[ \t]++([^\n]*)
FORMAT     = $1::$2
MV_ADD     = true

This means that anything with the source of WinEventLog:somthing will be run through the three field extracting transforms wel-message, wel-eq-kv and wel-col-kv. In that order.

wel-message splits the event into two fields, _pre_msg and Message.
wel-eq-kv splits the _pre_msg into field/value pairs based on 'equals' (=).
wel-col-kv splits the Message into field/value pairs based on colons (:).

However the REGEX in wel-col-kv requires that a newline preceeds the first capture group, and that newline does not exist in your first row.

Perhaps you could/should create new props.conf/transforms.conf stanzas in the /$SPLUNK_HOME/etc/system/local directory, which would be very similar, but the differently named. The props-stanza (from which the transforms are called) should be more specific, i.e. [WinEventLog:ErrorLogs], and the transforms that are called through the REPORT should be called welel-message, welel-eq-kv and welel-col-kv respectively (. Then the REGEX in welel-col-kv should have the leading newline made optional (by a question mark);

 SOURCE_KEY = Message
 REGEX      = \n?([^:\n\r]+):[ \t]++([^\n]*)
 FORMAT     = $1::$2
 MV_ADD     = true

Hope this helps, at least a little bit.


Ultra Champion

So it's pretty much like all the other WinEventLogs, i.e. Message contains everything from the Message= to the end of the event?

The answer is going to be to long for a comment. See answer below.


0 Karma

Path Finder

Message itself contains all of ApplicationName through StackTrace inside that field in the eventlog. A splunk search on this yields that the Severity, Error, and StackTrace Fields show up in the 22 interesting fields, I assume being autoextracted, but ApplicationName itself does not get extracted.

0 Karma

Ultra Champion

How does it not 'extract well'?
Is there a field called ApplicationName?
Does it contain anything?
Too much? Too little?

What is the value of the Message field?


0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...