Getting Data In

Cannot transfer rsyslog to Splunk.

sunrise
Contributor

May be it's easy but I got bogged down. Please help me.

I want to transfer rsyslog(hostA) to splunk(hostB) in TCP.
So I set the following configurations but I can't.

( inputs.conf@hostB )
[tcp://2000]
connection_host = dns
sourcetype = syslog

( rsyslog.conf@hostA )
*.info;mail.none;authpriv.none;cron.none @@hostB:2000

I confirmed /etc/hosts file is correct, and port 2000 is "LISTEN" state,
and both firewall is down.

I think that Splunk is waiting as a TCP receiver and some syslogd transer the logs to that.
Is it wrong ?
Other configuration is needed ?

Thank you for helping.

Tags (2)
0 Karma
1 Solution

jgedeon120
Contributor

On the indexer set up rsyslog or syslog-ng and configure it to listen on tcp 2000 also configure syslog to write to files for the indexer to read those files. This is actually the preferred method over receiving on a port using Splunk.

It's not that hard to configure and you can also filter easier.

http://www.rsyslog.com/receiving-messages-from-a-remote-system/

You can then have multiple systems set syslog to that server and written to files to be monitored.

A basic example in syslog-ng receiving would be:

source s_net { tcp(port(2000)) };
filter f_filtername { set(filters_here); };
destination d_filterdest { file("/var/log/remote/$HOST/sourcetype/log.log"); };
log {
source(s_net);
filter(f_filtername);
destination(d_filterdest);
};

On the Splunk indexer the monitor example:

.../local/inputs.conf

[monitor:///var/log/*/sourcetype/log.log]
disabled = 0
sourcetype = your_sourcetype
host_segment = 3
index = your_index

View solution in original post

0 Karma

sunrise
Contributor

Thank you, everyone. I've solved the problem.
The environments details that CentOS6.2 (hostA, hostB), and rsyslogd 5.8.10.
I edit /etc/sysconfig/rsyslog like below.

SYSLOGD_OPTIONS="-c 5 -f /etc/rsyslog.conf"

And I can transfer logs with changed TCP port number.
Thank you for your advice.

0 Karma

jgedeon120
Contributor

On the indexer set up rsyslog or syslog-ng and configure it to listen on tcp 2000 also configure syslog to write to files for the indexer to read those files. This is actually the preferred method over receiving on a port using Splunk.

It's not that hard to configure and you can also filter easier.

http://www.rsyslog.com/receiving-messages-from-a-remote-system/

You can then have multiple systems set syslog to that server and written to files to be monitored.

A basic example in syslog-ng receiving would be:

source s_net { tcp(port(2000)) };
filter f_filtername { set(filters_here); };
destination d_filterdest { file("/var/log/remote/$HOST/sourcetype/log.log"); };
log {
source(s_net);
filter(f_filtername);
destination(d_filterdest);
};

On the Splunk indexer the monitor example:

.../local/inputs.conf

[monitor:///var/log/*/sourcetype/log.log]
disabled = 0
sourcetype = your_sourcetype
host_segment = 3
index = your_index

0 Karma

sunrise
Contributor

Sorry, this may be caused by a rsyslog client configuration.
There are nothing captured by tcpdump (port 2000) on hostB.
Telnet connections from hostA to hostB is able.

I know that UF is better than TCP transfer, but not always able to install.....

I confirmed that TCP/514 is able to transfer logs, but other ports is not.

0 Karma

OL
Communicator

I would actually recommand to install a UF on host A that reads the rsyslog locally and send it to HostB...

0 Karma

MuS
SplunkTrust
SplunkTrust

in addition to Ayn's suggestion: have to tried to connect to hostB's port 2000 by using on hostA 'telnet hostB 2000' for example?

Ayn
Legend

Have you run tcpdump or similar on hostB to check that you're actually receiving data on port TCP/2000?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...