Getting Data In

How to edit my log4j sourcetype configuration on my Splunk forwarder for proper line breaking?

marshallmat
New Member

I have a java app that writes to a log file... I have configured a Splunk forwarder to forward this log (using source type 'log4j' ) to our Splunk indexer (central server located in different location)... things are fine except for searching, which displays multiple events for, what I consider one event. Example of the one 'event' in the log:

2016-04-29 21:46:26,760 INFO  AwsSqsReader:215 - .doTheRead()... MESSAGE BREADCRUMB... TYPE=XXXXX    MessageId:     mmmmm    ReceiptHandle: ddddddd    MD5OfBody:     3893393d93835e123d243903b84cfb34    Body:          {
  "Type" : "Notification",
  "MessageId" : "dddddd",
  "TopicArn" : "arn:aws:sns:us-east-1:dasfdasfdsafdasf:asdfasdfasdf",
  "Subject" : "ddddddddddd",
  "Message" : "{ \"EventCode\": 12345,         \"Summary\": \"llllll\",         \"Node\": \"uuuuuuu\",         \"Severity\": 3,         \"EMSInstance\": \"ddddddd\",         \"Agent\": \"aaaaaaa\",         \"AlertGroup\": \"ddd\",         \"AlertKey\": \"ddddd\",         \"Location\": \"ssssssssss\",         \"EpochTime\": \"1461980781\",         \"GenericString1\": \"abbbb\",         \"GenericString2\": \"12345\"         } ",
  "Timestamp" : "2016-04-30T01:46:22.278Z",
  "SignatureVersion" : "1",
  "Signature" : "xxxxxxxx",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-ddddddd",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=xxxxxxxxx"
}

There are two events shown on the search results page, with the split at the '"Timestamp":...' line.

On the splunk-forwarder host, I have tried to create my own sourcetype, reconfiguring the splunk-forwarder to use 'mylog4j' for this file... and then in $SPLUNKHOME/system/local/props.conf:

[mylog4j]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2},\d{3})
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25

I restarted the splunk-forwarder, sent a new event thru to the log, tried to re-search. Alas, I still see two events on search results page. I think the sourcetype change worked, since I had to change 'sourcetype=mylog4j' on the search.

How can I remedy this?

thx in advance
marshall

0 Karma

Jeremiah
Motivator

If you are running a universal forwarder, you need to put this props.conf setting on the indexer.

Get Updates on the Splunk Community!

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

What's New in Splunk Observability Cloud and Splunk AppDynamics - May 2025

This month, we’re delivering several new innovations in Splunk Observability Cloud and Splunk AppDynamics ...