Getting Data In

How to edit my log4j sourcetype configuration on my Splunk forwarder for proper line breaking?

marshallmat
New Member

I have a java app that writes to a log file... I have configured a Splunk forwarder to forward this log (using source type 'log4j' ) to our Splunk indexer (central server located in different location)... things are fine except for searching, which displays multiple events for, what I consider one event. Example of the one 'event' in the log:

2016-04-29 21:46:26,760 INFO  AwsSqsReader:215 - .doTheRead()... MESSAGE BREADCRUMB... TYPE=XXXXX    MessageId:     mmmmm    ReceiptHandle: ddddddd    MD5OfBody:     3893393d93835e123d243903b84cfb34    Body:          {
  "Type" : "Notification",
  "MessageId" : "dddddd",
  "TopicArn" : "arn:aws:sns:us-east-1:dasfdasfdsafdasf:asdfasdfasdf",
  "Subject" : "ddddddddddd",
  "Message" : "{ \"EventCode\": 12345,         \"Summary\": \"llllll\",         \"Node\": \"uuuuuuu\",         \"Severity\": 3,         \"EMSInstance\": \"ddddddd\",         \"Agent\": \"aaaaaaa\",         \"AlertGroup\": \"ddd\",         \"AlertKey\": \"ddddd\",         \"Location\": \"ssssssssss\",         \"EpochTime\": \"1461980781\",         \"GenericString1\": \"abbbb\",         \"GenericString2\": \"12345\"         } ",
  "Timestamp" : "2016-04-30T01:46:22.278Z",
  "SignatureVersion" : "1",
  "Signature" : "xxxxxxxx",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-ddddddd",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=xxxxxxxxx"
}

There are two events shown on the search results page, with the split at the '"Timestamp":...' line.

On the splunk-forwarder host, I have tried to create my own sourcetype, reconfiguring the splunk-forwarder to use 'mylog4j' for this file... and then in $SPLUNKHOME/system/local/props.conf:

[mylog4j]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2},\d{3})
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25

I restarted the splunk-forwarder, sent a new event thru to the log, tried to re-search. Alas, I still see two events on search results page. I think the sourcetype change worked, since I had to change 'sourcetype=mylog4j' on the search.

How can I remedy this?

thx in advance
marshall

0 Karma

Jeremiah
Motivator

If you are running a universal forwarder, you need to put this props.conf setting on the indexer.

Get Updates on the Splunk Community!

Get Operational Insights Quickly with Natural Language on the Splunk Platform

In today’s fast-paced digital world, turning data into actionable insights is essential for success. With ...

What’s New in Splunk Observability Cloud – June 2025

What’s New in Splunk Observability Cloud – June 2025 We are excited to announce the latest enhancements to ...

Almost Too Eventful Assurance: Part 2

Work While You SleepBefore you can rely on any autonomous remediation measures, you need to close the loop ...