Getting Data In

Why is some of the data that is coming in to Splunk via UDP getting dropped?

caili
Path Finder

Every UDP packet is like this below:

<headinfo product="wf" hash="D95F-7C1A-0F4D-A311" msgtype="3840" sip="0"/>
<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

There are two lines, but I want to combine to one event

Splunk indexes the data like this:

<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

the first line is dropped.

What can I do for this?
Thanks in advance.

Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

If the first line is indexed but not as the same event, then try this in your props to combine the events:

[sourcetypeName]
...
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = <headinfo

If the first event is being dropped altogether, then this may be due to UDP's connection-less design. It could also be due to other options set in your props & transforms.

0 Karma

caili
Path Finder

The first line is missing, not be indexed

if I add BREAK_ONLY_BEFORE = <headinfo in props.conf,then all the data(less than 256) will be combined as one envent

the configuration of props.conf is :

[a_log]
category = Custom
description = receiving UDP port data 
pulldown_type = 1
#SHOULD_LINEMERGE = true
#BREAK_ONLY_BEFORE = <headinfo

There is no transforms.conf in local directory

0 Karma

jkat54
SplunkTrust
SplunkTrust

Did you try it and all data less than 256 characters was combined into one event, or are you saying thats what will happen if you do try it?

should <headinfo... be one event and <wsec... be another event or do you want all of these to be one event?

0 Karma
Get Updates on the Splunk Community!

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...