AMQP Messaging Modular Input: How do we configure a RabbitMQ server with a universal forwarder?

[cdhawke1]

We are testing out an implementation of Splunk.

We are trying to have our logs flow from an internally hosted server to a RabbitMQ server to Splunk.

i.e. Universal Forwarder > RabbitMQ > Splunk (AMQP app).

Does this make sense? We've had conflicting sources as to whether or not this is even possible.

We are having quite a bit of trouble figuring out how to configure all of these apps to talk to each other properly.
We assumed that RabbitMQ was listening by default on 5672, so we set up the universal forwarder to talk to it on that port and we get some errors. We haven’t even gotten to the step where we try to configure AMQP in Splunk as we can’t get the logs to flow into a RabbitMQ queue.

Could you provide some insight here? I’m not very familiar with these configurations. The reason behind this is that we have a corporate firewall and we are trying to flow logs from inside the firewall to a Splunk server hosted in a different core. The RabbitMQ server is meant to facilitate this.

We are going to talk to our network admin to see if he will allow an exception, but the rule up to this point has been to not allow traffic to flow in that direction. We figured that if we could PULL from the queue using the AMPQ pp for Splunk, that we could then bypass the need to PUSH using a universal forwarder through the firewall.

Sincerely,
AMPQ Noob

[Damien_Dallimor]

1) To get log data from your source server to a RabbitMQ queue requires a RabbitMQ client.Without knowing anything about your server logging I can't advise whether there is any existing client you can use (such as a JMS logging appender/handler), or whether you'd need to write something custom for your scenario.

2) once the log data is successfully getting written to your RabbitMQ queue then it is pretty trivial to setup the AMQP Modular Input to read data from this queue. Here is a sample configuration, of course you'd need to replace param values to match your setup.

[amqp://testingamqp]
ack_messages = 1
exchange_name = amqp.splunk
hostname = localhost
index = main
index_message_envelope = 1
index_message_propertys = 1
password = guest
port = 5672
queue_name = splunkqueue
sourcetype = amqp
use_ssl = 0
username = guest
disabled = 1
basic_qos_limit = 20

[lucasfbeinjamin]

@Damien Dallimore what folder i put this config file and what is the name of this config file that i'll create?