topic Why are my udp syslog input events getting merged? in Getting Data In

Why are my udp syslog input events getting merged?

Chris_R_ — Fri, 26 Mar 2010 05:31:08 GMT

Odd behaviour with some udp syslog input from a Panorama device (palo alto management device) and ArcSight connector using the same udp input port. Input settings as follows

[udp://515]
disabled = false
sourcetype = threat_events
index = myindex

props.conf
[sourcetype::threat_events]
TZ = UTC
SHOULD_LINEMERGE = False

Linemerge settings were set to false for the sourcetype as seen above, however events merged at together sometimes. Is this an example of getting the right LINE_BREAKER= correct for the device?

Re: Why are my udp syslog input events getting merged?

Chris_R_ — Fri, 26 Mar 2010 05:32:34 GMT

One workaround was to use separate network ports for the different devices, curious to hear the answer though.

Re: Why are my udp syslog input events getting merged?

gkanapathy — Fri, 26 Mar 2010 09:30:53 GMT

Probably. LINE_BREAKER by default is ([\r\n]+), that is, any sequence of newlines and carriage returns. In addition, the end of a UDP packet will also end an event. It's not clear to my why you would have merged events in this case. Was there any pattern or commonality to the merged events?

Re: Why are my udp syslog input events getting merged?

Chris_R_ — Fri, 26 Mar 2010 23:57:02 GMT

They were getting merged events where a arcsight event would come through and multiple panorama event (starting with CEF) would get merged one after another ( client info removed )

Mar 4 18:47:25 somehost.com Mar 4 18:48:06 1,2010/03/048:48:06,0004A100609,THREAT,url,46,2010/03/04 18:48:05,10.170.133.122,82.195.186.201,0.0.0.0,0.0.0.0,ProxyAccess-A2,,,web-browsing,xtxs1,Int-FW,Int-FW-Proxy blah....de....blah....informational,0 (<---first event should end here)
CEF:0|Palo Alto|Panorama|||THREAT|Unknown| eventId=1428238 proto=UDP art=1267728483290 rt=1267728481000 shost=somehost.com src=10.97.3.55 sourceZoneURI=/zzz Zones/System Zones/Private Address Space dst=22.11.22.33 blah ....de...blah...dtz=Asia/xyz deviceFacility=IPS (<--- Second event should end here)
CEF:0|Palo Alto|Panorama (<---they continued to get multiple CEF Panorama device events all merged with the above)

Re: Why are my udp syslog input events getting merged?

gkanapathy — Sat, 10 Apr 2010 23:21:50 GMT

look to me then that SHOULD_LINEMERGE = false wasn't taking effect. (And therefore default BREAK_ONLY_BEFORE_DATE = true was in effect.) I always use lower-case false instead of upper-case False. I have no idea if that makes a difference.

Re: Why are my udp syslog input events getting merged?

jrodman — Sat, 17 Apr 2010 07:20:55 GMT

Use a stanza name:

[threat_events]

not

[sourcetype::threat_events]

We don't have any support for that syntax, and it shouldn't be in the docs anywhere (I can see how you might think there would be, of course).

Thus you should have:

[threat_events]
TZ = UTC
SHOULD_LINEMERGE = False