Multiline events not line breaking as expected

[lisheridan]

I have some data that looks like:

TIMESTAMP: 2011-10-31 13:51:25
top - 13:51:25 up 6 days, 19:53, 5 users, load average: 21.00, 20.57, 19.83
Tasks: 130 total, 0 running, 130 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.5% us, 0.7% sy, 0.0% ni, 96.4% id, 1.3% wa, 0.0% hi, 0.1% si
Mem: 32906264k total, 32847544k used, 58720k free, 346852k buffers
Swap: 33615352k total, 6804k used, 33608548k free, 7764416k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25772 admin 17 0 22.8g 22g 7040 S 19.9 72.1 10:17.11 rfsd
25780 admin 16 0 22.8g 22g 7040 S 19.9 72.1 10:18.01 rfsd
25777 admin 16 0 22.8g 22g 7040 S 17.9 72.1 10:18.10 rfsd
25459 admin 16 0 22.8g 22g 7040 S 11.9 72.1 8:40.27 rfsd
25493 admin 16 0 22.8g 22g 7040 S 6.0 72.1 2:03.05 rfsd

TIMESTAMP: 2011-10-31 13:52:25
top - 13:52:25 up 6 days, 19:53, 5 users, load average: 21.00, 20.57, 19.83
Tasks: 130 total, 0 running, 130 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.5% us, 0.7% sy, 0.0% ni, 96.4% id, 1.3% wa, 0.0% hi, 0.1% si
Mem: 32906264k total, 32847544k used, 58720k free, 346852k buffers
Swap: 33615352k total, 6804k used, 33608548k free, 7764416k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25772 admin 17 0 22.8g 22g 7040 S 19.9 72.1 10:17.11 rfsd
25780 admin 16 0 22.8g 22g 7040 S 19.9 72.1 10:18.01 rfsd
25777 admin 16 0 22.8g 22g 7040 S 17.9 72.1 10:18.10 rfsd
25459 admin 16 0 22.8g 22g 7040 S 11.9 72.1 8:40.27 rfsd
25493 admin 16 0 22.8g 22g 7040 S 6.0 72.1 2:03.05 rfsd

I want to line break only before "TIMESTAMP". Here is my props.conf:

[source::/var/log/stats/rfsd_top*]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = TIMESTAMP
MAX_EVENTS = 400

I sometimes get an event with just the "TIMESTAMP.." line while other times I get the correct event intact. The event size is 132 lines. How can I get this to work?

[lisheridan]

Unfortunately neither of those worked but thx for trying (even with SHOULD_LINEMERGE = true while using BREAK_ONLY_BEFORE).

[lguinn2]

To your original stanza, try adding

TRUNCATE = 40000

MAX_TIMESTAMP_LOOKAHEAD = 42

[_d_]

Give this stanza a try:

[source::/var/log/stats/rfsd_top*]
TIME_PREFIX = ^TIMESTAMP:\s+
TIME_FORMAT= %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
LINE_BREAKER = ([\r\n]+)(?=^TIMESTAMP:\s+\d{4}\-\d{2}\-\d{2})
SHOULD_LINEMERGE = false

Hope this helps

> please upvote and accept answer if you find it useful - thanks!

[gkanapathy]

[source::/var/log/stats/rfsd_top*]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)TIMESTAMP:
TRUNCATE = 40000
TIME_PREFIX = ^TIMESTAMP:
TIME_FORMAT = %Y-%m-%d %H:%M:%S

[gkanapathy]

yes, thank you. corrected above.

[kristian_kolb]

You're right, gkanapathy probably made a small mistake. Replace BREAK_ONLY_BEFORE with LINE_BREAKER.

/kristian

[lguinn2]

Shouldn't it be

SHOULD_LINEMERGE = true

if you are going to use BREAK_ONLY_BEFORE?