Getting Data In

Push logs from rsyslog into splunk

bhavya_shah
Path Finder

I was able to setup rsyslog to push logs into splunk but issue is only /var/log/messages are pushed to splunk but i have many more logs such as /logs/server-logs/servername/* on rsyslog server that I want to push it to splunk. Is there a way to push it?

Tags (2)
0 Karma
1 Solution

Joanna
New Member

Hi Bhavya,

What add-ons did you need on Splunk enterprise to receive logs from rsyslog client?

Was rsyslog on an external system? 

Thanks.

Joanna.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

In case you haven't noticed, this is a really old thread. Your question might not get the visibility you want in this thread. Try starting a new thread describing your problem in the Getting Data In section of this forum.

0 Karma

grijhwani
Motivator

In its simplest form you just need something like the following stanza in the inputs.conf on the rsyslog server. (I assume, from the mention of other logs already being pushed, you have installed a light forwarder instance at the very least.)

[monitor:///logs/server-logs/]
host_segment = 3
sourcetype = syslog
index = syslog
disabled = false

As splunk try the following:

bin/splunk btool inputs list --debug

This should show you your complete inputs configuration.

To interrogate what is or is not being consumed, point this at your indexer:

https://{yoursplunkserver}:8089/services/admin/inputstatus/TailingProcessor:FileStatus

It will show you what is and what is not being processed, and why.

bhavya_shah
Path Finder

Firewall issue has been fixed but in splunkd.log I am seeing following error:

07-15-2013 04:09:36.647 -0700 INFO TailingProcessor - Could not send data to output queue (parsingQueue), retrying...
07-15-2013 04:09:38.935 -0700 INFO TailingProcessor - ...continuing.

07-15-2013 04:10:00.879 -0700 INFO BatchReader - Removed from queue file=

07-15-2013 09:27:38.824 -0700 INFO WatchedFile - Will begin reading at offset

0 Karma

grijhwani
Motivator

In that case you seem to have firewall or routing issues.

0 Karma

bhavya_shah
Path Finder

I am still not getting logs but I have more issue.

/opt/splunkforwarder/bin/splunk list forward-server

Active forwards:
None
Configured but inactive forwards:
servername:9997

Here is the exact error on my forwarder on splunkd.log

07-11-2013 16:19:56.153 -0700 WARN TcpOutputProc - Cooked connection to ip=ipaddress:9997 timed out

0 Karma

bhavya_shah
Path Finder

I tried and it does show me complete inputs.conf info but still dont know why its not pushing the log.

0 Karma

bhavya_shah
Path Finder

Even though I added stanza in inputs.conf file I still dont see logs coming into splunk. I am not sure what I am missing here.

0 Karma

adrianathome
Communicator

You need to create an inputs.conf file on your forwarder that has a stanza for all the logs that you want.

bhavya_shah
Path Finder

If you dont mind can you share the example inputs.conf file?

0 Karma
Get Updates on the Splunk Community!

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

What's New in Splunk Observability Cloud and Splunk AppDynamics - May 2025

This month, we’re delivering several new innovations in Splunk Observability Cloud and Splunk AppDynamics ...