topic Re: Why is Splunk line breaking a single IDS Alert event into two events? in Getting Data In

Why is Splunk line breaking a single IDS Alert event into two events?

dmenon — Tue, 29 Sep 2020 09:23:17 GMT

Splunk is breaking ids single event into two events, such as:

4/11/16
2:42:46.152 PM 
04/11-14:42:46.152985 00:05:00:00:00:00 -> 00:00:00:05:00:01 type:0x800 len:0x222
10.20.30.40:59406 -> 106.120.151.145:80 TCP TTL:52 TOS:0x0 ID:53190 IpLen:20 DgmLen:532 DF
***A**** Seq: 0xBA0195C4  Ack: 0xBB15F92D  Win: 0x3E00  TcpLen: 20
[Xref => http://doc.emergingthreats.net/2008500]
host = ISMeta2 source = /var/log/snort/snort.log

4/11/16
2:42:46.000 PM 
[**] [1:2008500:6] ET MALWARE Sogoul.com Spyware User-Agent (SogouIMEMiniSetup) [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
host = ISMeta2 source = /var/log/snort/snort.log

Which appears in snort.log as this one event:

[**] [1:2008500:6] ET MALWARE Sogoul.com Spyware User-Agent (SogouIMEMiniSetup) [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
04/11-14:42:46.152985 00:05:00:00:00:00 -> 00:00:00:05:00:01 type:0x800 len:0x222
10.20.30.40:59406 -> 106.120.151.145:80 TCP TTL:52 TOS:0x0 ID:53190 IpLen:20 DgmLen:532 DF
***A**** Seq: 0xBA0195C4  Ack: 0xBB15F92D  Win: 0x3E00  TcpLen: 20
[Xref => http://doc.emergingthreats.net/2008500]

All events start with [**] I have props.conf configured as follows where snort_alert_full is sourcetype, but that doesn't fix my issue

[snort_alert_full]      
BREAK_ONLY_BEFORE = [**]

Thanks in advance

Re: Why is Splunk line breaking a single IDS Alert event into two events?

martin_mueller — Tue, 29 Sep 2020 09:20:27 GMT

BREAK_ONLY_BEFORE expects a regular expression, [**] would be extremely malformed for your data. Have you tried \[\*\*\]?

That being said, you'd be better off utilizing LINE_BREAKER like this:

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\[\*\*\]
TIME_PREFIX = ^([^\r\n]+[\r\n]+){2}
MAX_TIMESTAMP_LOOKAHEAD = 25
TIME_FORMAT = %m/%d-%H:%M:%S.%6N

That'll be MUCH faster and should achieve exactly what you need.

Re: Why is Splunk line breaking a single IDS Alert event into two events?

dmenon84 — Tue, 12 Apr 2016 14:43:08 GMT

Thanks for the help, this did not work, I think there is more to the way the logs are setup as well. For instance - I want splunk to look at event starting with [**] and then grab the timestamp from the third line of the log. How do I achieve that ?

[**] [1:2008500:6] ET MALWARE Sogoul.com Spyware User-Agent (SogouIMEMiniSetup) [**]
 [Classification: A Network Trojan was Detected] [Priority: 1]
 04/11-14:42:46.152985 00:05:00:00:00:00 -> 00:00:00:05:00:01 type:0x800 len:0x222
 10.20.30.40:59406 -> 106.120.151.145:80 TCP TTL:52 TOS:0x0 ID:53190 IpLen:20 DgmLen:532 DF
 ***A**** Seq: 0xBA0195C4  Ack: 0xBB15F92D  Win: 0x3E00  TcpLen: 20
 [Xref => http://doc.emergingthreats.net/2008500]

Thanks in advance.

Re: Why is Splunk line breaking a single IDS Alert event into two events?

dmenon84 — Tue, 12 Apr 2016 16:46:23 GMT

Some more sample events:

Event 1

[**] [1:2520169:2541] ET TOR Known Tor Exit Node UDP Traffic group 85 [**]
[Classification: Misc Attack] [Priority: 2]
04/12-11:22:01.201114 00:00:00:05:00:01 -> 00:05:00:00:00:00 type:0x800 len:0x53
93.158.215.174:23320 -> 90.80.70.60:53 UDP TTL:52 TOS:0x28 ID:43018 IpLen:20 DgmLen:69
Len: 41
[Xref => http://doc.emergingthreats.net/bin/view/Main/TorRules]

Event 2

[**] [1:2016104:3] ET TROJAN DNS Reply for unallocated address space - Potentially Malicious 1.1.1.0/24 [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
04/12-11:30:31.770737 00:00:00:05:00:01 -> 00:05:00:00:00:00 type:0x800 len:0x69
125.39.136.74:53 -> 30.40.50.60:56038 UDP TTL:51 TOS:0x0 ID:35728 IpLen:20 DgmLen:91
Len: 63

and my current props.conf because I see [**} at 2 places so I expanded on your suggestion.

[snort_alert_full]
SHOULD_LINEMERGE = false
LINE_BREAKER =[**]\s+[\d+:\d+:\d+]

Re: Why is Splunk line breaking a single IDS Alert event into two events?

martin_mueller — Tue, 12 Apr 2016 17:26:35 GMT

The LINE_BREAKER from my answer works fine using your sample event from the question. I've added timestamp parsing to the answer.

Re: Why is Splunk line breaking a single IDS Alert event into two events?

martin_mueller — Tue, 12 Apr 2016 17:28:54 GMT

That LINE_BREAKER cannot work. First, it doesn't have a capturing group for the linebreaker processor to consume and second, you're not escaping the regex special chars asterisk and square bracket.

Re: Why is Splunk line breaking a single IDS Alert event into two events?

dmenon84 — Wed, 13 Apr 2016 16:46:29 GMT

Hi, I think your answer will definitely work but I don't think I have the props.conf in the right place basically the logs are being forwarded to splunk through universal forwarder. Under etc/deployment-apps I see a folder SplunkUFSnortconfig folder which is where the inputs.conf is configured as to which location to monitor name of index,sourcetype etc. I created props.conf in SplunkUFSnortconfig -> local but that doesn't seem to do anything at all.

Re: Why is Splunk line breaking a single IDS Alert event into two events?

martin_mueller — Wed, 13 Apr 2016 19:34:20 GMT

These props.conf settings need to be where the parsing phase happens, that'll be a heavy forwarder or the indexers in this case. The universal forwarder will ignore these.