How do i stop a file from being segmented?

[MikeBertelsen]

This is the beginning of the file, line numbers for clarity:
1. Log File for: BatchJobOutput_20150801-0139_13516_MonthlyBatchJob_SAMM191.log
2. Started: Sat Aug 1 01:39:22 CDT 2015
3. Using path to access.properties: /opt/WebSphere/AppServer/lib/app
4. --------------------------
5. /usr/java64/jdk1.6.0_43/bin:/bin:/usr/bin:/opt/gnome/bin:/usr/X11/bin:/home/cd7543/scripts

This is the end of the file:
159. Ended: Sat Aug 1 01:40:55 CDT 2015
160.

There are many date references between these two sections and Splunk takes this one file and splits it up which is then displayed to the end user segmented, in reverse order.

How do I get Splunk to index this as one contiguous file?

[somesoni2 · Accepted solution]

Try something like this for your props.conf

[YourSourceType]
BREAK_ONLY_BEFORE=^Log\s*File\s*for:
MAX_TIMESTAMP_LOOKAHEAD=150
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
TIME_PREFIX=Started:\s*

[MikeBertelsen]

Of the solutions offered, this one works the best.
But I am exceeding the 10000 byte limit and the file is being truncated.
Suggestions?

[somesoni2]

10000 bytes limits in the preview screen? If you see data truncated in Preview, don't worry actual data will not get truncated.
If you see the actual data getting truncated, add following to props.conf stanza.

TRUNCATE = 0

[martin_mueller]

Do not set settings such as TRUNCATE to zero, ever.

Pick a value that seems outlandishly large, sure - but not zero. If something breaks at your source, Splunk will attempt to build infinitely large events...

[MikeBertelsen]

yes in the preview screen

[somesoni2]

You want all 160 lines to appear into one big single event with timestamp being taken from line 2?

[MikeBertelsen]

Yes that is correct.

[woodcock]

Add this to your props.conf:

TIME_PREFIX = ^Started:\s+

This won't work if the other timestamps start with the same RegEx.

[MikeBertelsen]

That helps a lot. the file is now broken up into two segments. the first line and the second which contains the timestamp.
I can make do with this but if needed how do I get the previous line included?

[woodcock]

Add these, too:

 BREAK_ONLY_BEFORE_DATE = false
 BREAK_ONLY_BEFORE = ^Log File for:\s+
 SHOULD_LINEMERGE = true