Getting Data In

Need help with props on multiline event

banderson7
Communicator

We're bringing in syslog's from datapower units, and they have a rough log setup:

Jul 22 09:00:20 10.214.8.104 [0x80c0003f][xxxxxxSSOSPDebug][info] xmlfirewall(SSOAuditLogFW): trans(214401)[request][100.73.230.24] gtid(214401):
Jul 22 09:00:20 10.214.8.104 Timestamp(2016-07-22T09:00:20-04:00)
Jul 22 09:00:20 10.214.8.104                            ::
Jul 22 09:00:20 10.214.8.104                            TransactionID()
Jul 22 09:00:20 10.214.8.104                            ::
Jul 22 09:00:20 10.214.8.104                            ClientId(xxxxxx 4000025)
Jul 22 09:00:20 10.214.8.104                            ::
Jul 22 09:00:20 10.214.8.104                            UserInfo()
Jul 22 09:00:20 10.214.8.104                            ::
Jul 22 09:00:20 10.214.8.104                            Status(0x00000000)
Jul 22 09:00:20 10.214.8.104                            ::
Jul 22 09:00:20 10.214.8.104                            TimeTake(V1_113_103_0_1_9)
Jul 22 09:00:20 10.214.8.104                            ::
Jul 22 09:00:20 10.214.8.104                            Message()

That's an event in syslog, but sometimes breaks in half in splunk:

Jul 21 23:15:44 172.22.93.21                ::
Jul 21 23:15:44 172.22.93.21                TimeTake(V1_207_187_0_0_20)
Jul 21 23:15:44 172.22.93.21                ::
Jul 21 23:15:44 172.22.93.21                Message()


Jul 21 23:15:44 172.22.93.21 [0x80c0003f][xxxxxxSSOSPDebug][info] xmlfirewall(SSOAuditLogFW): trans(33153921)[request][216.105.103.106]: 
Jul 21 23:15:44 172.22.93.21 Timestamp(2016-07-21T23:15:44-04:00)
Jul 21 23:15:44 172.22.93.21                ::
Jul 21 23:15:44 172.22.93.21                TransactionID()
Jul 21 23:15:44 172.22.93.21                ::
Jul 21 23:15:44 172.22.93.21                ClientId(xxxxxx 4000025)
Jul 21 23:15:44 172.22.93.21                ::
Jul 21 23:15:44 172.22.93.21                UserInfo()
Jul 21 23:15:44 172.22.93.21                ::
Jul 21 23:15:44 172.22.93.21                Status(0x00000000)

My props looks like this, and I'm pretty sure it's wrong, but not certain where:

[xxxxxx:prod:datapower]
#Dec 28 11:21:03 10.1.1.49 [cert-monitor][warn] cert-monitor(Certificate Monitor): trans(383): Certificate 'HummingbirdUATCert' is expired
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 15
#LINE_BREAKER = ([\n\r]+)(?=\w{3}\s+\d+\s+\d+:\d+:\d+)
LINE_BREAKER = ^(?:[^ \n]* ){5}(?P<reg>\[\d+\w+\])
TRUNCATE = 1024000
SHOULD_LINEMERGE = TRUE

Can someone help me stop these from coming in broken in half? I'm also interested in extracting the ClientId and Message fields, but w/ the amount of lines, am not getting far.

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try

[xxxxxx:prod:datapower]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=(\S+\s){4}\[)
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 15
TRUNCATE = 1024000

View solution in original post

0 Karma

mhassan
Path Finder

Can you explain where does the single event start and stop? What are the logical boundaries around an event? I see timestamp in every line (which in normal logs are single events) but they dont appears to be the case here.

0 Karma

banderson7
Communicator

This is a single event:
Jul 22 09:00:55 10.214.8.104 [0x80c0003f][xxxxxxSSOSPDebug][info] xmlfirewall(SSOAuditLogFW): trans(214449)[request][100.73.230.24] gtid(214449):
Jul 22 09:00:55 10.214.8.104 Timestamp(2016-07-22T09:00:55-04:00)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 TransactionID()
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 ClientId(xxxxxx 4000172)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 UserInfo()
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 Status(0x00000000)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 TimeTake(V1_180_169_0_1_10)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 Message()

0 Karma

somesoni2
Revered Legend

Give this a try

[xxxxxx:prod:datapower]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=(\S+\s){4}\[)
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 15
TRUNCATE = 1024000
0 Karma

banderson7
Communicator

That brought in the entire syslog file as 1 entry.
Edit: and the entries that are coming in now are correct, and being divided at the right lines. Wonder why the file came in as 1 entry.

0 Karma

somesoni2
Revered Legend

Could you explain more on the issue that you see?

0 Karma

banderson7
Communicator

Sure. I ingested my syslog file w/ that props and it came in as one event:
Jul 22 09:00:20 10.214.8.104 [0x80c0003f][xxxxxxSSOSPDebug][info] xmlfirewall(SSOAuditLogFW): trans(214401)[request][100.73.230.24] gtid(214401):
Jul 22 09:00:20 10.214.8.104 Timestamp(2016-07-22T09:00:20-04:00)
Jul 22 09:00:20 10.214.8.104 ::
Jul 22 09:00:20 10.214.8.104 TransactionID()
Jul 22 09:00:20 10.214.8.104 ::
Jul 22 09:00:20 10.214.8.104 ClientId(xxxxxx 4000025)
Jul 22 09:00:20 10.214.8.104 ::
Jul 22 09:00:20 10.214.8.104 UserInfo()
Jul 22 09:00:20 10.214.8.104 ::
Jul 22 09:00:20 10.214.8.104 Status(0x00000000)
Jul 22 09:00:20 10.214.8.104 ::
Jul 22 09:00:20 10.214.8.104 TimeTake(V1_113_103_0_1_9)
Jul 22 09:00:20 10.214.8.104 ::
Jul 22 09:00:20 10.214.8.104 Message()
Jul 22 09:00:55 10.214.8.104 [0x80c0003f][xxxxxxSSOSPDebug][info] xmlfirewall(SSOAuditLogFW): trans(214449)[request][100.73.230.24] gtid(214449):
Jul 22 09:00:55 10.214.8.104 Timestamp(2016-07-22T09:00:55-04:00)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 TransactionID()
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 ClientId(xxxxxx 4000172)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 UserInfo()
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 Status(0x00000000)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 TimeTake(V1_180_169_0_1_10)
Jul 22 09:00:55 10.214.8.104 ::
Jul 22 09:00:55 10.214.8.104 Message()
Jul 22 09:01:56 10.214.8.104 [0x80c0003f][xxxxxxSSOSPDebug][info] xmlfirewall(SSOAuditLogFW): trans(214513)[request][100.73.230.24] gtid(214513):
Jul 22 09:01:56 10.214.8.104 Timestamp(2016-07-22T09:01:56-04:00)
Jul 22 09:01:56 10.214.8.104 ::
Jul 22 09:01:56 10.214.8.104 TransactionID()
Jul 22 09:01:56 10.214.8.104 ::
Jul 22 09:01:56 10.214.8.104 ClientId(xxxxxx 4000192)
Jul 22 09:01:56 10.214.8.104 ::
Jul 22 09:01:56 10.214.8.104 UserInfo()
Jul 22 09:01:56 10.214.8.104 ::
Jul 22 09:01:56 10.214.8.104 Status(0x00000000)
Jul 22 09:01:56 10.214.8.104 ::
Jul 22 09:01:56 10.214.8.104 TimeTake(V1_151_138_0_0_13)
Jul 22 09:01:56 10.214.8.104 ::
Jul 22 09:01:56 10.214.8.104 Message()
with another 2000 lines. New events coming into splunk are broken up correctly, so the regex is correct. I'm uncertain how to break up the first event, and am uncertain if I want to try, depending on what's involved. Can you suggest any way to extract the ClientId and message fields?

0 Karma

somesoni2
Revered Legend

Try this for field extract
For Message Message\(\)([\r\n]*)(\S+\s+){4}(?<Message>.+)
For ClientId ClientId\((?<ClientId>[^\)]+)

0 Karma

somesoni2
Revered Legend

Doubt, so new events are working fine, then which ones are the one failing and where (splunk preview tool or actual data ingestion).?

0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...