All Apps and Add-ons

How to configure JMS Modular Input on a heavy forwarder to receive messages from a remote Active MQ JMS Queue?

lyndac
Contributor

I would like to use the JMS Modular Input to receive messages from a remote ActiveMQ JMS Queue. My scenario is this:

I have a splunk environment with 1 search head, 2 indexers, and a Universal Forwarder. I would like to add a Heavy Forwarder to this environment (on the same machine as the UF, if that is possible). The Heavy Forwarder would host the JMS Modular Input.

The only information I have about the JMS source I am connecting to is that it is activeMQ and I know the host, port, user, password and queue name. So this is where I get confused. I'm not sure if I'm supposed to do the JNDI thing or the local instantiation. Of course, I want to do whatever is the easiest and fastest. But I am not familiar with JNDI and also am not sure what would go into the local instantiation. Do I need to set up and run some sort of JNDI service on my host? Or is it as easy as giving the right combination of the info I have to the JMS Modular input UI?

I apologize if my questions seem broad. I am reading up on JNDI but it doesn't seem to be clicking. Any help would be appreciated.

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You can try to fill in the values in the setup based off this config I have used that works.

Here is an example input stanza for ActiveMQ. Use your actual environment values for the items inside of asterisks.

    [jms://queue/dynamicQueues/**splunkqueue**]
    host = **foo**
    index = main
    sourcetype = **foo**
    init_mode = jndi
    jms_connection_factory_name = **ConnectionFactory**
    jndi_initialcontext_factory = org.apache.activemq.jndi.ActiveMQInitialContextFactory
    jndi_provider_url = **tcp://somehostname:61616**
    disabled = 0
    strip_newlines = 0
    index_message_header = 1
    index_message_properties = 1

Make sure you have placed the ActiveMQ jar file in the jms_ta/bin/lib directory.

alt text

Also , follow any troubleshooting steps in the documentation.

View solution in original post

0 Karma

Damien_Dallimor
Ultra Champion

You can try to fill in the values in the setup based off this config I have used that works.

Here is an example input stanza for ActiveMQ. Use your actual environment values for the items inside of asterisks.

    [jms://queue/dynamicQueues/**splunkqueue**]
    host = **foo**
    index = main
    sourcetype = **foo**
    init_mode = jndi
    jms_connection_factory_name = **ConnectionFactory**
    jndi_initialcontext_factory = org.apache.activemq.jndi.ActiveMQInitialContextFactory
    jndi_provider_url = **tcp://somehostname:61616**
    disabled = 0
    strip_newlines = 0
    index_message_header = 1
    index_message_properties = 1

Make sure you have placed the ActiveMQ jar file in the jms_ta/bin/lib directory.

alt text

Also , follow any troubleshooting steps in the documentation.

View solution in original post

0 Karma

lyndac
Contributor

Thank for all the help Damien. I had to use the Local Instantiation method rather than JNDI. I have a class that implements the Local JMSResourceFactory and things seem to be working ok.

0 Karma

gmelasecca
Engager

lyndac/All - i know this post is older but can you post your splunk config? we dont use JNDI at my work either from what im understanding or at least i'm not sure where to find the information for the below:

[jms://queue/dynamicQueues/**splunkqueue**]

--when you say splunk queue name, what is this referring to exactly?
jms_connection_factory_name = ConnectionFactory
--unsure where i can find the connectionFactory that works. i have looked through every config in ActiveMQ
jndi_initialcontext_factory = org.apache.activemq.jndi.ActiveMQInitialContextFactory
--unsure where i can find the initialcontext_factory that works. i have looked through every config in ActiveMQ

0 Karma

lyndac
Contributor

Below is my .conf file for my JMS queue connection. The queue name in the stanza is the name of the queue that you are going to read from. You will need to get that from your ActiveMQ admin person.

The other thing to note is that I had to write two java classes -- one to implement the context factory, and one as a message handler. You will need to build a jar file containing your classes & dependencies and put it in the jms_ta/bin/lib directory. (Please excuse any typos...I had to retype it as I am on a closed system) I hope this helps

[jms://queue/:nameOfQueueToReadFrom]
browse_frequency=30
browse_queue_only=0
destination_pass=****
destination_user=system
durable=0
index=foo
index_message_header = 0
index_message_properties=0
init_mode=local
local_init_mode_resource_factory_impl = mycompany.jmssplunk.MyResourceFactory
local_init_mode_resource_factory_params=serverURL="tcp://7.123.12.44:61616,userName=system,password=password
message_handler_impl=`mycompany.jmssplunk.MyMessageHandler`
message_selector=type=summary
strip_newlines=1
sourcetype=foo-bar
disable=1

This is the java class for the initial context:

package mycompany.jmssplunk;

import com.splunk.modinput.jms.LocalJMSResourceFactory;
import javax.jms.*;
import org.apache.activemq.ActiveMQConnectionFactory;

public class MyResourceFactory implements LocalJMSResourceFactory {
  private String serverURL;
  private String userName;
  private String password;

  @Override
  public void setParams(Map<String,String> params) throws Exception {
    try {
      this.serverURL=params.get("serverURL");
      this.userName=params.get("userName");
      this.password=params.get("password");
    } catch (Throwable e) {
      throw new Exception("Error setting parameters for resource factory - " + e.getMessage();
    }
  }

  @Override
  public Queue createQueue (String queueName) throws Exeption {
    Session session = null;
    try {
       ConnectionFactory factory = createConnectionFactory();
       Connection connection = factory.createConnection(userName, password);
        session = connection.createSession(false, javax.jms.Session.AUTO_ACKNOWLEDGE);
         Queue queue = session.createQueue(queueName);
          return queue;
    } catch (Throwable e) {
         throw new Exception("Error creating Queueu - " + e.getMessage();
    }
  }

  @Override
   public ConnectionFactory createConnectionFactory() throws Exception{
      try {
          ConnectionFactory factory = new ActiveMQConnectionFactory(serverURL);
           return factory;
      } catch (Throwable e) {
         throw new Exception("Error creating Queueu - " + e.getMessage();
      }
    }
}
0 Karma

lyndac
Contributor

So is the jms_connection_factory supposed to be implemented by the owner of the ActiveMQ then? The Queue I'm connecting to does not have JNDI endpoints so I guess that means I need to do the local instantiation.

0 Karma

Damien_Dallimor
Ultra Champion

I think you need to talk to your ActiveMQ admin. They will give you the settings for your environment.

And read this : http://activemq.apache.org/jndi-support.html

And then also actually try out what I provided.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!