Why is my line break configuration not working for forwarded data, but works fine for local data in Splunk 6.2?

[rsawant]

I have a WebLogic *.out log file which has multiple lines (upto 500) in some of the events. When I indexed a sample of this on a test splunk instance, it works fine. I gave the following parameters:

MAX_EVENTS = 2560
NO_BINARY_CHECK = true
TIME_FORMAT = %b %d, %Y %T %p
TIME_PREFIX = <
BREAK_ONLY_BEFORE_DATE=true
disabled = false
pulldown_type = true

Now, when I forward the files to the production splunk instance, line break is not working properly. The MAX_EVENTS parameter seems to be read properly since I am getting linecounts greater than 257 for some events. I can't seem to figure out why the events are not breaking properly in case of forwarded data. I used btool to check the props configurations and things seem fine there. Please HELP!!!!

[jeff]

I'm just throwing some stuff out here:

Are you using a Universal Forwarder or a full Splunk heavy forwarder? If you're using full Splunk, the data gets "cooked" before it gets sent to the indexer. In this event, you need to either put the relevant props.conf configs on the forwarders, or set sendCookedData=false in your outputs.conf file and let the config take hold at the indexer. From a performance perspective, since you're (theoretically) using heavy forwarders it'd be best to do the parsing at the forwarder to relieve the indexer...

If you're already using a Universal Forwarder, the props.conf needs to be at the indexer where line breaking and other phases of data parsing happens. This is the typical modern deployment, since the Universal Forwarder is more light-weight and will have less impact on your endpoints.

See: http://docs.splunk.com/Documentation/Splunk/6.2.5/Forwarding/Typesofforwarders

[somesoni2]

Q1) In PROD did you put the props.conf on Indexers?
Q2) Can you provide some sample events?

[p_gurav]

I have same problem.... Can anyone please suggest something??? I have been debugging since two days........Please HELP!!!!!!!

[rsawant]

PS: The test instance, production instance as well as forwarder are using splunk 6.2