Why JMS Messaging Modular Input removes carriage return (0x0d) character during indexing?

[hannus]

I'm having hard time dealing with carriage return in Splunk Enterprise!

Indexing a file with CR+LF at the end of each line (on Windows) using Add Data wizard in the main view works perfectly when using the "LINE_BREAKER=((*FAIL))" control. But when I use the JMS Messaging Modular Input with that control and the same props.conf stanza, it does not work. Indexing messages using this modular input removes the CR character at the end of each line. LF character is not removed. The event with multiple lines looks nice on the Search app but the CR is missing if I look at the "0" file in the "rawdata" folder.

I do not have the option "Strip newline characters from message body" selected so the CR and LF should be indexed.

How can I fix this ?

Basically I'm trying to import data in to Splunk "as is" and eventually export the event also "as is". Exporting also removes the CR character, by the way...

Thanks for any help!

[Damien_Dallimor]

Post your inputs.conf stanza for your JMS input and any props.conf and transforms.conf stanzas you are applying.

[hannus]

I'm importing messages that are in xml format and I'm extracting 25 fields during indexing. I verified from the messaging system that it is sending the CR character. Everything else seems to work fine. I managed to get rid of the jms message header with the fresh version of the JMS Messaging Modular Input (v.1.5.1). That was good!

inputs.conf (in /apps/launcher/local):

[jms://queue/:QSPLUNKIN_Dest]
browse_frequency = 30
browse_mode = all
browse_queue_only = 0
durable = 0
hec_batch_mode = 0
hec_https = 0
index = jms
index_message_header = 0
index_message_properties = 0
init_mode = jndi
jms_connection_factory_name = SplunkConnectionFactory
jndi_initialcontext_factory = com.sun.jndi.fscontext.RefFSContextFactory
jndi_provider_url = file:/C:/MQJNDI
output_type = stdout
sourcetype = ME120_st_spec
strip_newlines = 0
disabled = 0
message_handler_impl = com.splunk.modinput.jms.custom.handler.BodyOnlyMessageHandler

props.conf (in apps/jms_ta/local):

[ME120_st_spec]
NO_BINARY_CHECK = true
category = Custom
description = My comment here
pulldown_type = 1
disabled = false
MAX_TIMESTAMP_LOOKAHEAD = 19
TIME_FORMAT = %Y-%m-%dT%H:%M:%S
TIME_PREFIX = < MonitoringTime > ((<-- had to add spaces here to show the text))
MAX_EVENTS = 50000
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^ < ? xml version ((<-- also had to add spaces here to show the text))
TRUNCATE = 60000
TRANSFORMS-me120 = Field1,Field2,Field3,Field4,...,Field24,Field25
LINE_BREAKER = ((*FAIL))

transforms.conf (in apps/jms_ta/local):

[Field1]
REGEX = ((?<=Field1>).*?(?=< / Field1>)) ((<-- again more spaces here to show text))
FORMAT = Field1::$1
WRITE_META = true

[Field2]
REGEX = ((?<=Field2>).*?(?=< / Field2>)) ((<-- and again more spaces here to show text))
FORMAT = Field2::$1
WRITE_META = true

...[Field25]...

Thanks for your help!