Getting Data In

Multiline events not line breaking as expected

lisheridan
Explorer

I have some data that looks like:

TIMESTAMP: 2011-10-31 13:51:25
top - 13:51:25 up 6 days, 19:53, 5 users, load average: 21.00, 20.57, 19.83
Tasks: 130 total, 0 running, 130 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.5% us, 0.7% sy, 0.0% ni, 96.4% id, 1.3% wa, 0.0% hi, 0.1% si
Mem: 32906264k total, 32847544k used, 58720k free, 346852k buffers
Swap: 33615352k total, 6804k used, 33608548k free, 7764416k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25772 admin 17 0 22.8g 22g 7040 S 19.9 72.1 10:17.11 rfsd
25780 admin 16 0 22.8g 22g 7040 S 19.9 72.1 10:18.01 rfsd
25777 admin 16 0 22.8g 22g 7040 S 17.9 72.1 10:18.10 rfsd
25459 admin 16 0 22.8g 22g 7040 S 11.9 72.1 8:40.27 rfsd
25493 admin 16 0 22.8g 22g 7040 S 6.0 72.1 2:03.05 rfsd

TIMESTAMP: 2011-10-31 13:52:25
top - 13:52:25 up 6 days, 19:53, 5 users, load average: 21.00, 20.57, 19.83
Tasks: 130 total, 0 running, 130 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.5% us, 0.7% sy, 0.0% ni, 96.4% id, 1.3% wa, 0.0% hi, 0.1% si
Mem: 32906264k total, 32847544k used, 58720k free, 346852k buffers
Swap: 33615352k total, 6804k used, 33608548k free, 7764416k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25772 admin 17 0 22.8g 22g 7040 S 19.9 72.1 10:17.11 rfsd
25780 admin 16 0 22.8g 22g 7040 S 19.9 72.1 10:18.01 rfsd
25777 admin 16 0 22.8g 22g 7040 S 17.9 72.1 10:18.10 rfsd
25459 admin 16 0 22.8g 22g 7040 S 11.9 72.1 8:40.27 rfsd
25493 admin 16 0 22.8g 22g 7040 S 6.0 72.1 2:03.05 rfsd

I want to line break only before "TIMESTAMP". Here is my props.conf:

[source::/var/log/stats/rfsd_top*]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = TIMESTAMP
MAX_EVENTS = 400

I sometimes get an event with just the "TIMESTAMP.." line while other times I get the correct event intact. The event size is 132 lines. How can I get this to work?

Tags (1)
0 Karma

lisheridan
Explorer

Unfortunately neither of those worked but thx for trying (even with SHOULD_LINEMERGE = true while using BREAK_ONLY_BEFORE).

0 Karma

lguinn2
Legend

To your original stanza, try adding

TRUNCATE = 40000

MAX_TIMESTAMP_LOOKAHEAD = 42

0 Karma

_d_
Splunk Employee
Splunk Employee

Give this stanza a try:

[source::/var/log/stats/rfsd_top*]
TIME_PREFIX = ^TIMESTAMP:\s+
TIME_FORMAT= %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
LINE_BREAKER = ([\r\n]+)(?=^TIMESTAMP:\s+\d{4}\-\d{2}\-\d{2})
SHOULD_LINEMERGE = false

Hope this helps

> please upvote and accept answer if you find it useful - thanks!

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
[source::/var/log/stats/rfsd_top*]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)TIMESTAMP:
TRUNCATE = 40000
TIME_PREFIX = ^TIMESTAMP:
TIME_FORMAT = %Y-%m-%d %H:%M:%S

gkanapathy
Splunk Employee
Splunk Employee

yes, thank you. corrected above.

0 Karma

kristian_kolb
Ultra Champion

You're right, gkanapathy probably made a small mistake. Replace BREAK_ONLY_BEFORE with LINE_BREAKER.

/kristian

0 Karma

lguinn2
Legend

Shouldn't it be

SHOULD_LINEMERGE = true

if you are going to use BREAK_ONLY_BEFORE?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...