Getting Data In

How to edit my log4j sourcetype configuration on my Splunk forwarder for proper line breaking?

marshallmat
New Member

I have a java app that writes to a log file... I have configured a Splunk forwarder to forward this log (using source type 'log4j' ) to our Splunk indexer (central server located in different location)... things are fine except for searching, which displays multiple events for, what I consider one event. Example of the one 'event' in the log:

2016-04-29 21:46:26,760 INFO  AwsSqsReader:215 - .doTheRead()... MESSAGE BREADCRUMB... TYPE=XXXXX    MessageId:     mmmmm    ReceiptHandle: ddddddd    MD5OfBody:     3893393d93835e123d243903b84cfb34    Body:          {
  "Type" : "Notification",
  "MessageId" : "dddddd",
  "TopicArn" : "arn:aws:sns:us-east-1:dasfdasfdsafdasf:asdfasdfasdf",
  "Subject" : "ddddddddddd",
  "Message" : "{ \"EventCode\": 12345,         \"Summary\": \"llllll\",         \"Node\": \"uuuuuuu\",         \"Severity\": 3,         \"EMSInstance\": \"ddddddd\",         \"Agent\": \"aaaaaaa\",         \"AlertGroup\": \"ddd\",         \"AlertKey\": \"ddddd\",         \"Location\": \"ssssssssss\",         \"EpochTime\": \"1461980781\",         \"GenericString1\": \"abbbb\",         \"GenericString2\": \"12345\"         } ",
  "Timestamp" : "2016-04-30T01:46:22.278Z",
  "SignatureVersion" : "1",
  "Signature" : "xxxxxxxx",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-ddddddd",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=xxxxxxxxx"
}

There are two events shown on the search results page, with the split at the '"Timestamp":...' line.

On the splunk-forwarder host, I have tried to create my own sourcetype, reconfiguring the splunk-forwarder to use 'mylog4j' for this file... and then in $SPLUNKHOME/system/local/props.conf:

[mylog4j]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2},\d{3})
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25

I restarted the splunk-forwarder, sent a new event thru to the log, tried to re-search. Alas, I still see two events on search results page. I think the sourcetype change worked, since I had to change 'sourcetype=mylog4j' on the search.

How can I remedy this?

thx in advance
marshall

0 Karma

Jeremiah
Motivator

If you are running a universal forwarder, you need to put this props.conf setting on the indexer.

Get Updates on the Splunk Community!

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...

Major Splunk Upgrade – Prepare your Environment for Splunk 10 Now!

Attention App Developers: Test Your Apps with the Splunk 10.0 Beta and Ensure Compatibility Before the ...

Stay Connected: Your Guide to June Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...