All Apps and Add-ons

Data not ingesting into splunk from RabbitMQ queue

getmesomedata
Explorer

I'm having some issues trying to get my data from my RabbitMQ instance into splunk.

I've completed the following steps:
- Enabled the STOMP protocol in my installation of RabbitMQ
- Installed the STOMP app in my spunk instance and setup a data input to listen to my queue (127.0.0.1\topic\testQueue)
- Published some messages onto the queue which results in no data in splunk.

I've checked the list of connections within RabbitMQ and there is a connection from splunk so I know that part has worked. I've checked the splunk internal errors and I can't see anything relating to the STOMP app.

Can you suggest any other logs for me to check or is there anything obvious I've missed out?

0 Karma
1 Solution

allenta
Explorer

And the 'mysterious' buffering issue is now fixed! Please, upgrade to v0.3 and check if your problem persists.

Thank you!

View solution in original post

0 Karma

allenta
Explorer

And the 'mysterious' buffering issue is now fixed! Please, upgrade to v0.3 and check if your problem persists.

Thank you!

0 Karma

allenta
Explorer

Great 🙂

The issue was trivial. A forgot flush call in the stream which connects the modular input and Splunk. A beginner's mistake.

0 Karma

getmesomedata
Explorer

Success, v0.3 works a charm! Thanks

Out of curiosity what was the issue?

0 Karma

allenta
Explorer

Hi getmesomedata!

The steps you've followed are perfectly correct. It would be helpful if you can make a quick test in order to check if the issue you're experiencing is related with a strange behaviour we are still researching.

We've detected some kind of event buffering somewhere in between RabbitMQ and Splunk. Due to that 'mysterious' buffering, if you test the STOMP modular input with only a few messages, they arrive at Splunk, but they are never rendered in the UI until the buffer is completely filled. So, please, repeat your test with 100 or more messages (you can use the producer.py script in https://github.com/allenta/splunk-stomp/tree/master/extras/clients if you want). Let us know if that way you are able to see the enqueued messages in the Splunk Search UI.

Thank you for the report!

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...