Hi All
We want to index multiline log messages with no timestamp as one event.
But regular expression for multiline is difficult.
So now I try following configurations in props.conf.
[Test_buildmasterlog]
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z
EXTRACT-bid = (?m)^BuildID:(?P<bid>.+)^Release:
EXTRACT-release = (?m)^Release:(?P<release>.+)[\r\n]
EXTRACT-environment = ^Environment:(?P<environment>.+)^BuildType:
EXTRACT-buildtype = (?m)^BuildType:(?P<buildtype>.+)
EXTRACT-starttime = (?m)^Start Time:(?P<starttime>.+)^Status:
EXTRACT-status = (?m)^Status:(?P<status>.+)^Elapsed Time:
EXTRACT-elapsedtime = (?m)^Elapsed Time:(?P<elapsedtime>.+)
And the log is look like that
BuildID:20140818-00
Release:R20.5
Environment:QA3
BuildType:Daily Auto Build
Start Time:2014-08-18 11:00:00 000
Status:Completed
Elapsed Time:25mins
However when I do a searching using the following query
sourcetype="test_buildmasterlog"|table bid,release,environment, buildtype, starttime, status,elapsedtime
It will separate to two rows. So which place that I set wrongly?
Your props.conf
is specifically telling Splunk that your events are 1 line each. That's not what you want! Instead of this
[Test_buildmasterlog]
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z
Try this
[Test_buildmasterlog]
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE = ^BuildID
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z
The rest of props.conf
is fine.
Your props.conf
is specifically telling Splunk that your events are 1 line each. That's not what you want! Instead of this
[Test_buildmasterlog]
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z
Try this
[Test_buildmasterlog]
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE = ^BuildID
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z
The rest of props.conf
is fine.
Thanks Lguinn. It's work.
And one more question. Look at your conf, you define only the keyword of "BuildID" for break the line only. if I don't know which field is the first appear in the log. How can I do?
One would expect entries in the same log file to follow a standard pattern. The "break_only_before" is predicated on that assumption. There is only so much one can achieve with configurations. To some extent it is necessary for the input source to conform to some reasonable standard of expected behaviour.
@Chrismok: As from your props file i dont see any configuration that tells splunk to index multiple lines together.
Could you please provide log sample.