Why are multiple events being merged together as a single event?

[lohitkidu]

Hi,

I have following logs which are being merged as a single event in Splunk. There are similar events which are correctly parsed by Splunk, but some of them are not.

2016-01-27 15:55:49.009064+00:00 name="http" event_id=http:notifier src=xx.xx.xx.xx dest="xx" dest_port=tcp/80 url="/xx" http_method="GET" http_user_agent="xx" http_refer="xx"
2016-01-27 15:55:49.016709+00:00 name="http" event_id=http:notifier src=xx dest="xx" dest_port=tcp/80 url="xx" http_method="GET" http_user_agent="xx" http_refer="xx"

and following is the props.conf for this:

[source::/var/log/abc/...]
TRANSFORMS-SK=a,b,c
TIME_PREFIX=^
TIME_FORMAT=%y-%m-%d %H:%M:%S.%6N%z
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = false

Any ideas on this?

[alemarzu]

Just use this on your props.conf & btw I had a typo on my first answer and already fixed it. This should work as expected.

[source::/var/log/abc/...]
 BREAK_ONLY_BEFORE = \d{4}-\d{2}-\d{2}\s
 SHOULD_LINEMERGE = true
 TIME_PREFIX=^
 TIME_FORMAT=%y-%m-%d %H:%M:%S.%6N%z
 MAX_TIMESTAMP_LOOKAHEAD = 32

Hope this is clear now.

[alemarzu]

Try adding this to your props.conf

BREAK_ONLY_BEFORE = \d{4}-\d{2}-\d{2}\s
SHOULD_LINEMERGE = true

[lohitkidu]

Should this work even on my indexed fields ? because currently it is not

[lohitkidu]

** indexed events I meant above.

[alemarzu]

What was already bad indexed wont get re-parsed. You should probably delete those and re-index.

[lohitkidu]

As the issue is not replication too much, so cannot test it. But i didnot get the issue itself as these events are well formed with breaklines \r\n. Splunk can learn these events breaks automatically. Also why is it occurrence in some events and not others.

[alemarzu]

Can you paste another data sample of those events that are not getting parsed correctly ?

Run this at CLI and search for the affected sourcetype stanza to debug.

 %SPLUNK_HOME$/bin>./splunk cmd btool props list

[lohitkidu]

well nothing wrong i can found in props for the given sourcetype.