Getting Data In

Windows Event Log Format and JSON

Path Finder

Hi,

I have developers who are trying to create a framework for Windows Event Error handling that can be used for any in-house developed application. They have decided that all errors will be logged into a custom windows event log. However, in the message field of the event they have decided to use JSON to describe the events details.

07/22/2013 02:26:45 PM
LogName=aaaa
SourceName=bbbb
EventCode=1000
EventType=2
Type=Error
ComputerName=cccc
User=dddd
Sid=eeee
SidType=1
TaskCategory=None
OpCode=Info
RecordNumber=66
Keywords=Classic
Message={"EventDateTime" : "2013-07-22T04:26:43Z",
"Message" : "A turbo integrator error has occurred",
"User" : "ffff",
"AdminHost" : "gggg",
"Server" : "hhhh",
"DataSource" : "iiii",
"DataSourceType" : "ODBC",
"ProcessStartDateTime" : "2013-07-22T14:26:42",
"LogFileURL" : "\\jjjjdev\Data$\kkkkdev\Logging\ProcessError20130722042643.log"}

How do I extract the Message value and parse it as JSON? or write the whole event as XML? Then there is the issue of working from within the Windows Event log schema which is not flexible enough to provide custom fields.

Thanks.

Tags (4)
0 Karma
1 Solution

Path Finder

Hi,

Did you try the spath function on the message field ? http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Spath

It should solve your problem.

View solution in original post

0 Karma

Path Finder

Hi,

Did you try the spath function on the message field ? http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Spath

It should solve your problem.

View solution in original post

0 Karma

Path Finder

Cool!
It is not recommended to create new fields at index time. The gain is real only for few cases. The reasons are well explained here : http://docs.splunk.com/Documentation/Splunk/5.0.3/Indexer/Indextimeversussearchtime
If you really want to create custom index fields, read this http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/Configureindex-timefieldextraction

0 Karma

Path Finder

Hi Michael,

I have similar data with Message field json in Windows Event. I am using spath to search the Message json but the problem is that Splunk by default parses the Message field as key value pairs so I end up with duplicate values. E.g
Message={
"description" : "Sample text",
"eventid" : "47",
"id" : "22",
"logtype" : "Error",
"msgnum" : "0",
"severity" : "Reserved",
"source" : "Sample source",
"status" : "New",
"system
state" : "S4/S5",
"timestamp" : "00-01-01 00:00:00",
"timestampaccuracy" : "Approximate"
}
For the above Message field Splunk already has parsed event
id with value "\"47\",". When I use spath and count by eventid Splunk adds 47 also to the events so I end up with duplicate eventids for each eventid (1, "1",), (2, "2",) etc.
Is there a way to explicitly turn of Splunk parsing so that I can parse Message in the search (| spath input=Message | stats count by event
id)

0 Karma

Path Finder

Hi michael. Thanks that works. It has extracted fields from the Message field using : as delimiters. Do you think I could do the same thing at index time using props.conf? Is it worth doing at index time?

0 Karma