topic Re: still splitting on blank line in Getting Data In

still splitting on blank line

tgiles — Wed, 04 Apr 2012 16:54:04 GMT

Hi, All.

I'm trying to parse trend micro logs on a windows system using a heavy forwarder. Running into issues getting the entries to split in the right spot.

* Date Time:[20120404 00:10:46]
* Server Information: 
        Server Name:[myserver.com]  IP:[192.168.0.1]  Port:[80]
* Client Information: 
        Computer Name:[SERVER1]  Port:[38292]
        Client IP:[10.200.0.200  ] 
        GUID:[af697491-b868-48d3-b868-fa13854d48d3] 
* Action: Scheduled Update
* Result: Scheduled Update, Started

=========================================================================================

Splunk splits on ([\r\n]+), so something like the above will end up making two event entries in Splunk when only one actual entry is made to the log file itself.

I've defined the TIME_FORMAT and TIME_PREFIX attempted to BREAK_ONLY_BEFORE_DATE, but that didn't work.
I've configured the MUST_BREAK_AFTER to the long "===" line, but it didn't.
I've set the MUST_NOT_BREAK_AFTER to the default break regex, but it still broke in the same place.
Also attempted to use a SEDCMD to remove the long "===" line completely, but it didn't work, either.

I'm running out of really good ideas on how to force the entire entry into just one Splunk event. Mind pointing me in the right direction on this?

Thanks!

tom

Re: still splitting on blank line

kristian_kolb — Wed, 04 Apr 2012 17:04:05 GMT

I'd suggest that you alter your props.conf on the Heavy Forwarder accordingly;

[your_sourcetype_whatever_it's_called]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+={10,100})

This should use the newline(s), followed by anything between 10 and 100 '='-characters as the break between events. There should be no MUST_BREAK.., BREAK_ONLY.. etc, but you could/should still have the TIME_FORMAT / TIME_PREFIX / MAX_TIMESTAMP_LOOKAHEAD.

Hope this helps,

Kristian

Re: still splitting on blank line

tgiles — Fri, 06 Apr 2012 18:07:51 GMT

Thanks for the response, Kristian. Unfortunately, the logs are still getting split on the blank line- creating two entries.

Will keep reviewing to see if there's an alternate method available to this.

Re: still splitting on blank line

tgiles — Fri, 06 Apr 2012 19:27:18 GMT

In the end, I used a SEDCMD just to remove the errant extra line that Splunk was splitting on. In the props.conf, I added:

[host::*]
SEDCMD-avlog = s/=========================================================================================//g

That works.