LINEMERGE / props.conf issues on XML

helius — Tue, 16 Dec 2014 14:32:36 GMT

Greetings,

I'm receiving XML files and noticed that the first half (around 257 lines) is a single event, but everything after that is a separate event.

Reading the documentation, I see mention of modifying the props.conf to fix this. So, I've decided to use the following:

[source::elemental_job]
SHOULD_LINEMERGE = true
MAX_EVENTS = 2000
BREAK_ONLY_BEFORE = ^(<command)
MUST_NOT_BREAK_BEFORE = ^(</command)

However, this does not work. It seems to have no effect actually.

Can someone tell me what I'm doing wrong?

Re: LINEMERGE / props.conf issues on XML

helius — Fri, 02 Jan 2015 17:08:57 GMT

After much research, found the solution......

I realized that the forwarders don't actually handle working with the data I see on the search head. This is done on the indexers. So I added the following to the props.conf on the indexers we use and it works perfectly.

[source::/opt/elemental_se/web/log/*/job_*/*_emecmd.xml]
SHOULD_LINEMERGE = true
TRUNCATE = 0
MAX_EVENTS = 5000
BREAK_ONLY_BEFORE = ^(<command)
MUST_NOT_BREAK_BEFORE = ^(</command)

topic LINEMERGE / props.conf issues on XML in Getting Data In

LINEMERGE / props.conf issues on XML

Re: LINEMERGE / props.conf issues on XML