Getting Data In

Why is some of the data that is coming in to Splunk via UDP getting dropped?

caili
Path Finder

Every UDP packet is like this below:

<headinfo product="wf" hash="D95F-7C1A-0F4D-A311" msgtype="3840" sip="0"/>
<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

There are two lines, but I want to combine to one event

Splunk indexes the data like this:

<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

the first line is dropped.

What can I do for this?
Thanks in advance.

Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

If the first line is indexed but not as the same event, then try this in your props to combine the events:

[sourcetypeName]
...
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = <headinfo

If the first event is being dropped altogether, then this may be due to UDP's connection-less design. It could also be due to other options set in your props & transforms.

0 Karma

caili
Path Finder

The first line is missing, not be indexed

if I add BREAK_ONLY_BEFORE = <headinfo in props.conf,then all the data(less than 256) will be combined as one envent

the configuration of props.conf is :

[a_log]
category = Custom
description = receiving UDP port data 
pulldown_type = 1
#SHOULD_LINEMERGE = true
#BREAK_ONLY_BEFORE = <headinfo

There is no transforms.conf in local directory

0 Karma

jkat54
SplunkTrust
SplunkTrust

Did you try it and all data less than 256 characters was combined into one event, or are you saying thats what will happen if you do try it?

should <headinfo... be one event and <wsec... be another event or do you want all of these to be one event?

0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...