Getting Data In

how to allow multiple source on udp 514 port

ebmadmin
New Member

we want to receive data from multiple devices on udp port 514, but splunk interface not allowing to second source on same port. is it limitation ?

0 Karma

PavelP
Motivator

Hello @ebmadmin ,

you can specify several inputs for the same port, but there are some restrictions:

[udp://<remote server>:<port>]
* Similar to the [tcp://] stanza, except that this stanza causes the Splunk
  instance to listen on a UDP port.
* Only one stanza per port number is currently supported.
* Configures the instance to listen on a specific port.
* If you specify <remote server>, the specified port only accepts data
  from that host.
* If <remote server> is empty - [udp://<port>] - the port accepts data sent
  from any host.
  * The use of <remote server> is not recommended. Use the 'acceptFrom'
    setting, which supersedes this setting.
* Generates events with source set to udp:portnumber, for example: udp:514
* If you do not specify a sourcetype, generates events with sourcetype set
  to udp:portnumber.

Also check this: https://docs.splunk.com/Documentation/Splunk/8.0.3/Data/Monitornetworkports
In your case you can configure several different inputs, but you need to specifiy a block for each udp source:

[udp://192.168.100.200:514]
index = firewall
sourcetype = checkpoint


[udp://192.168.200.50:514]
index = security
sourcetype = secure
0 Karma

vessev
Path Finder

Hello ebmadmin,

if you want to work with a more complex syslog input it's also best practise to use syslog-ng.
You can install it on a different linux vm all together with a universal forwarder. Or on the indexer itself but i would not reccomend that.

The syslog-ng config could look a little bit like that.
I for example has configured the syslog-ng to sort the input from different sources via syslog-ng:
In this example i used the filter to separate cisco-ios syslog from normal syslog.
You can do a lot with the filters - check the website since this tool is free and open-source.

/etc/syslog-ng/conf.d/syslog_udp514.conf

# Syslog configuration File: syslog_udp514.conf
#
# Should be placed under /etc/syslog-ng/conf.d/syslog_udp514.conf
# 
# Autor: vessev
# Date : somedate
# Note : Compatible with syslog-ng Version 3.26
#

options {
chain_hostnames(off);                                   
create_dirs (yes);          
dir_perm(0750);         
#dns_cache(yes);    
keep_hostname(yes);                 
log_fifo_size(1024);
#log_msg_size(8192);
perm(0644);     
time_reopen (10);
use_dns(yes);   
use_fqdn(yes);
group(syslog-ng);
perm(0640);
};

source s_udp514 { udp(port(514)); };

#Destinations
destination d_cisco_ios  { file("/data/syslog/forwarder/u514/$HOST/$YEAR$MONTH$DAY-cisco-ios.log" create_dirs(yes)); };
destination d_syslog_u514 { file("/data/syslog/forwarder/u514/$HOST/$YEAR$MONTH$DAY-syslog.log" create_dirs(yes)); };

# Filters
filter f_cisco_ios { match("(?:%|#)(?:(?!POLICY_ENGINE|UCSM|FWSM|ASA|FTD|PIX|ACE)[A-Z0-9_]+)" value("PROGRAM")) or match("(?:%|#)(?:(?!POLICY_ENGINE|UCSM|FWSM|ASA|FTD|PIX|ACE)[A-Z0-9_]+)" value("MESSAGE")); };
filter f_syslog_u514 { not (
filter(f_cisco_ios)
);
};

# Log
log { source(s_udp514); filter(f_cisco_ios);  destination(d_cisco_ios);  };
log { source(s_udp514); filter(f_syslog_u514); destination(d_syslog_u514); };

And then for a simple universal forwarder to send this data to your indexer, use this kind of inputs.conf:

vim /opt/splunkforwarder/etc/system/local/inputs.conf

# Unfiltered syslog
[monitor:///data/syslog/forwarder/u514/*/*syslog.log]
sourcetype = syslog
disabled = 0
host_segment = 5

# Cisco IOS
[monitor:///data/syslog/forwarder/u514/*/*cisco-ios.log]
sourcetype = syslog
disabled = 0
host_segment = 5

You can define multiple inputs if you're able to divide your streams with syslog-ng filters.
But i would recommend to use different ports for different products. Like UDP Port 5517 for Cisco ASA, Port 5518 for Palo Alto etc. with this separation you don't need to filter that much.

Best regards,

vess

0 Karma

ebmadmin
New Member

Ok... if we leave remote server empty , then though all sources can send in logs on same port but then how can we maintain seperate indexes for them ?

0 Karma

vessev
Path Finder

Okay let me explain it by showing you the difference a little bit more graphical:

Your infrastructure is looking a little bit like this at the moment:

LogSource -----------UDP/514---------> Splunk (Indexer)

You can like PavelP mentioned not make a general input but a "specific" input.
He means that you can type the source IP Address of your Log Source (like a switch or router etc.)
With that you can divide your syslog input. BUT: you would need for every LogSource a separate block. You can also give this a specific Index. Like (example PavelP):

 [udp://192.168.100.200:514]
 index = firewall
 sourcetype = checkpoint


 [udp://192.168.200.50:514]
 index = security
 sourcetype = secure

My example takes a different approach in comparison to pavelp:
I have an additional system which is collecting the Syslog First:

LogSource -----------UDP/514---------> SyslogForwarder ----------TCP/9997------> Splunk (Indexer)

The syslog forwarder in my example is a linus system. It has a splunk universal forwarder running which forwards every traffic to my splunk indexer. And also a syslog-ng installed (Tool for syslog collecting). The Syslog-ng collects any UDP/TCP Traffic like UDP 514 and stores it as Log File on the disk. The splunk universal forwarder looks into those files and forwards the events in those Log files to the splunk indexer.

Best regards, vess

0 Karma

vessev
Path Finder

You can order the splunk forwarder (Universal Forwarder on the SyslogForwarder) to catch the paths which syslog-ng uses to store the incomming UDP syslog packages.
You can also specify a index for those inputs.

vi /opt/splunkforwarder/etc/system/local/inputs.conf

[default]
host = SyslogForwarder

# Unfiltered syslog
[monitor:///path/to/logs/u514/*/*syslog.log]
sourcetype = syslog
index = syslog
disabled = 0
host_segment = 5

# Cisco IOS
[monitor:///path/to/logs/u514/*/*cisco-ios.log]
sourcetype = syslog
index = cisco
disabled = 0
host_segment = 5
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...