All Apps and Add-ons

How to convert byte characters coming out of MQ through JMS Messaging Modular input app

umapadhi
Explorer

Implemented JMS Messaging Modular input(JMS_TA )app to read a Websphere MQ. The message text is coming in with byte characters. I can read the field name fine but the message body is mostly byte characters. When we copy to notepad, it looks like below
msg_body=" \ufffd\ufffd\ufffd3\ufffd\ufffd\ufffd\ufffd\ufffdF\ufffd)\ufffd\ufffd\ufffd\ufffd\ufffd\ufffdF\ufffd"\ufffdu\ufffdS/S/S/S/S/S/S/S/\ufffdF\ufffd"\ufffd\ufffdow\ufffdS/ S/S/S/S/S/S/S/". Looking at the data still inside the queue looks ok. How should we conevrt the data from byte characters to text format?

1 Solution

Damien_Dallimor
Ultra Champion

As the received data is encoded in EBCDIC , which not understood by Splunk , you will need to write a custom message handler for the JMS Modular Input.

This custom message handler would receive the raw bytes , and translate them from EBCDIC to ASCII. You will have to find the IBM Java libraries to assist you with this translation.

1) Code your custom handler. Pseudo code example for you to follow and complete :

package com.foo;

import java.util.Map;

import javax.jms.Message;

import com.splunk.modinput.jms.AbstractMessageHandler;
import com.splunk.modinput.jms.JMSModularInput.MessageReceiver;

public class DecodeEBCDICHandler extends AbstractMessageHandler {

    @Override
    public void handleMessage(Message message, MessageReceiver context)
            throws Exception {


        String bodyContent = decodeEBCDIC(message);

        transportMessage(bodyContent);
    }

    private String decodeEBCDIC(Message message) {

        String decodedContent = "";

        //write some code to get the message content and decode it

        return decodedContent;
    }

    @Override
    public void setParams(Map<String, String> params) {
        // Do nothing , params not used

    }

}

2) Compile it and put it in a jar file
3) Drop the jar file in jms_ta/bin/lib
4) Declare the custom handler to be applied on the JMS Mod Input setup page for your stanza

alt text

View solution in original post

Damien_Dallimor
Ultra Champion

As the received data is encoded in EBCDIC , which not understood by Splunk , you will need to write a custom message handler for the JMS Modular Input.

This custom message handler would receive the raw bytes , and translate them from EBCDIC to ASCII. You will have to find the IBM Java libraries to assist you with this translation.

1) Code your custom handler. Pseudo code example for you to follow and complete :

package com.foo;

import java.util.Map;

import javax.jms.Message;

import com.splunk.modinput.jms.AbstractMessageHandler;
import com.splunk.modinput.jms.JMSModularInput.MessageReceiver;

public class DecodeEBCDICHandler extends AbstractMessageHandler {

    @Override
    public void handleMessage(Message message, MessageReceiver context)
            throws Exception {


        String bodyContent = decodeEBCDIC(message);

        transportMessage(bodyContent);
    }

    private String decodeEBCDIC(Message message) {

        String decodedContent = "";

        //write some code to get the message content and decode it

        return decodedContent;
    }

    @Override
    public void setParams(Map<String, String> params) {
        // Do nothing , params not used

    }

}

2) Compile it and put it in a jar file
3) Drop the jar file in jms_ta/bin/lib
4) Declare the custom handler to be applied on the JMS Mod Input setup page for your stanza

alt text

umapadhi
Explorer

Hi Damien,
Any update on this? How to convert those byte characters to ASCII? It has both texts and numbers.

Thanks
Uma

0 Karma

Damien_Dallimor
Ultra Champion

quick note : I'm tied up with splunk conf 15 this week , will look at this after the event.

0 Karma

rcbiswal
Engager

We could see the JMS_IBM_CHARACTER_SET is ISO-8859-1 and JMS_IBM_ENCODING is 273 (which is an EBCDIC code page) in the message properties. Will this answer to your query?

0 Karma

umapadhi
Explorer

Thanks Damien. I will check that.

0 Karma

Damien_Dallimor
Ultra Champion

What is the encoding ? EBCDIC ? Please ask your system admins and/or the developers that are publishing the messages to the the MQ queue.

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...