Why is linux collector server not reporting syslog to splunk instance?

[dminguez]

Hi,

We are trying to send syslogs from 3 different enpoints from different suppliers to an ubuntu 20.04 server.

I'm recieving this syslogs over UDP 514 port and trying to send them over the TCP port 9997 to the splunk instance, in order to be processed. I have installed the universal splunk forwarder targeting the host:port that I needed in the collector VM, although I'm not receiving any traffic from the firewalls and I get some logs from the collector VM with missing chunks of information.

I have checked that all communications and ports are up and responding, and the output.conf file has the right settings but, port 9997 is unavailable from the splunk web panel when I try to add it as data input in settings.

Does anyone know if I am missing anything here?

Do I need to use syslog-ng to successfully send syslogs to the splunk instance from a Linux VM?

Thanks for your help!

Regards.

[venky1544]

Hi @dminguez 

if you install syslog-ng on the linux VM and redirect the logs to a file and redirect the splunk UF to read the syslog-ng data from the file on the VM and send the data to splunk would be a splunk recommended best practise . Basically when  you log to a local file, it provides a local buffer and you aren't blocked if the network goes down. 

and when you say "port 9997 is unavailable from the splunk web panel"  port 9997 needs to be configured forwarding and recieving option it would not be under data input 

please follow below option

setting >> forwarding and recieving >>configure recieving 

[dminguez]

Thanks for the answer I configured the port under the "forwarding and recieving" option as you said and went through this documentation (https://www.splunk.com/en_us/blog/tips-and-tricks/using-syslog-ng-with-splunk.html). I was able to get the configuration that I needed to put in /syslog-ng/conf.d, although it doesn't write the directories in the destinations that are given. I also checked if SELinux was interfering but it is disabled, do you know what the problem could be?.

Thanks for your help.

Regards.

[venky1544]

Hi @dminguez 

so when you say "it doesn't write the directories in the destinations that are given" 

1) so you meant syslog is not writing the data to the directories or splunk UF is not reading the directory you need to share your syslog conf details 

2) is your VM(ubuntu)  receiving the data from the three different endpoints  how are you validating that ?? 

Thanks

[dminguez]

I meant that syslog wasn't writing the directories detailed in the config file but, I've managed to get all logs coming from the firewalls through 514 UDP, in a txt file.

I am monitoring that file using SplunkUF and I can see it in the splunk web panel although, I'm trying to use the script posted in the community in order to separate the logs into different files or folders and it isn't working.

Is there any update of the script available? if so, could anyone share it in this post?.

I'm going to share here the script that I've been using.

Thanks.

@version:3.2

# syslog-ng configuration file.
#
#
options {
chain_hostnames(no);
create_dirs (yes);
dir_perm(0755);
dns_cache(yes);
keep_hostname(yes);
log_fifo_size(2048);
log_msg_size(8192);
perm(0644);
time_reopen (10);
use_dns(yes);
use_fqdn(yes);
};

source s_network {
udp(port(514));
};

#Destinations
destination d_cisco_asa { file(“/home/syslog/logs/cisco/asa/$HOST/$YEAR-$MONTH-$DAY-cisco-asa.log” create_dirs(yes)); };
destination d_palo_alto { file(“/home/syslog/logs/paloalto/$HOST/$YEAR-$MONTH-$DAY-palo.log” create_dirs(yes)); };
destination d_all { file(“/home/syslog/logs/catch_all/$HOST/$YEAR-$MONTH-$DAY-catch_all.log” create_dirs(yes)); };

# Filters
filter f_cisco_asa { match(“%ASA” value(“PROGRAM”)) or match(“%ASA” value(“MESSAGE”)); };
filter f_palo_alto { match(“009401000570” value(“PROGRAM”)) or match(“009401000570” value(“MESSAGE”)); };
filter f_all { not (
filter(f_cisco_asa) or
filter(f_palo_alto)
);
};
# Log
log { source(s_network); filter(f_cisco_asa); destination(d_cisco_asa); };
log { source(s_network); filter(f_palo_alto); destination(d_palo_alto); };
log { source(s_network); filter(f_all); destination(d_all); };