Hello,
I have an application sending logs to the windows event log with a lognamename of ErrorLogs. The error log looks like this:
05/30/2012 03:17:42 PM
LogName=ErrorLogs
SourceName=LogDataManager
EventCode=0
EventType=4
Type=Information
ComputerName=DEVSERVICE
TaskCategory=%1
OpCode=Info
RecordNumber=31
Keywords=Classic
Message=ApplicationName: EventLogTestWriter
Severity: Informational
Error: Added newline before ApplicationName
StackTrace: N/A
All fields extract well in the autoextraction in search except for the first field 'ApplicationName:' Any ideas? I tried a new line break and while it does send ApplicationName to a new line I still don't get the field to extract.
All WinEvent logs go through the parsing rules defined in $SPLUNK_HOME/etc/system/local/props.conf
and $SPLUNK_HOME/etc/system/local/transforms.conf
.
These state the following;
props.conf:
[source::WinEventLog...]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv
KV_MODE=none
etc, etc
transforms.conf:
[wel-message]
REGEX = (?sm)^(?<_pre_msg>.+)\nMessage=(?<Message>.+)$
CLEAN_KEYS = false
[wel-eq-kv]
SOURCE_KEY = _pre_msg
DELIMS = "\n","="
MV_ADD = true
[wel-col-kv]
SOURCE_KEY = Message
REGEX = \n([^:\n\r]+):[ \t]++([^\n]*)
FORMAT = $1::$2
MV_ADD = true
This means that anything with the source of WinEventLog:somthing will be run through the three field extracting transforms wel-message
, wel-eq-kv
and wel-col-kv
. In that order.
wel-message
splits the event into two fields, _pre_msg
and Message
.
wel-eq-kv
splits the _pre_msg
into field/value pairs based on 'equals' (=).
wel-col-kv
splits the Message into field/value pairs based on colons (:).
However the REGEX in wel-col-kv requires that a newline preceeds the first capture group, and that newline does not exist in your first row.
Perhaps you could/should create new props.conf/transforms.conf
stanzas in the /$SPLUNK_HOME/etc/system/local
directory, which would be very similar, but the differently named. The props-stanza (from which the transforms are called) should be more specific, i.e. [WinEventLog:ErrorLogs], and the transforms that are called through the REPORT should be called welel-message
, welel-eq-kv
and welel-col-kv
respectively (. Then the REGEX in welel-col-kv
should have the leading newline made optional (by a question mark);
[welel-col-kv]
SOURCE_KEY = Message
REGEX = \n?([^:\n\r]+):[ \t]++([^\n]*)
FORMAT = $1::$2
MV_ADD = true
Hope this helps, at least a little bit.
Kristian
So it's pretty much like all the other WinEventLogs, i.e. Message contains everything from the Message= to the end of the event?
The answer is going to be to long for a comment. See answer below.
/k
Message itself contains all of ApplicationName through StackTrace inside that field in the eventlog. A splunk search on this yields that the Severity, Error, and StackTrace Fields show up in the 22 interesting fields, I assume being autoextracted, but ApplicationName itself does not get extracted.
How does it not 'extract well'?
Is there a field called ApplicationName?
Does it contain anything?
Too much? Too little?
What is the value of the Message field?
/k