I'm consuming a qa test log that has a fairly erratic format, but I was able to identify a line breaker regex to group them into events the way I want. It works perfectly if it consumes the entire file at once, but under normal conditions, a "single" event contains multiple messages that are written over a span of several minutes or longer as the qa test steps through its tasks. In the example below, that total time span is about 20 minutes. Under these conditions, Splunk is indexing each message as its own event no matter what I try.
Here's a sample event (snipped for brevity):
2010-Dec-09 16:19 - loading suite testKprSsl.rb
startup logic
switching fbc from http to https
got fbc.cfg
deploying new fbc.cfg updated with https
Loaded suite testKprSsl
Started
TestKprPrivileges:
test_100_ConfigureFbc:
.: (13.903755)
test_101_UpgradeFbc:
.: (13.932528)
test_102_QueryFbc:
.: (20.092743)
(snip)
Finished in 1248.147737289 seconds.
78 tests, 264 assertions, 2 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
97.4359% passed
Here is the props.conf entry I used that worked fine when indexing complete files - it looks for the particular date format at the start of a new line:
[watcherresults]
LINE_BREAKER = ([\r\n])+(\d+)-[A-Za-z]{3}-(\d+\s)
TIME_PREFIX = ^
TIME_FORMAT = %Y-%b-%d %H:%M
TRUNCATE = 0
I have also tried using different combos of BREAK_ONLY_BEFORE regexes, SHOULD_LINEMERGE, and BREAK_ONLY_BEFORE_DATE, but the messages are always showing up as multiple events.
The indexed file is written on a remote server and forwarded via LWF to a Windows 7 system running version 4.1.5, build 85165. Is there a way to keep these messages together as a single event or to reassemble them after the fact?
... View more