Getting Data In

How to edit my log4j sourcetype configuration on my Splunk forwarder for proper line breaking?

marshallmat
New Member

I have a java app that writes to a log file... I have configured a Splunk forwarder to forward this log (using source type 'log4j' ) to our Splunk indexer (central server located in different location)... things are fine except for searching, which displays multiple events for, what I consider one event. Example of the one 'event' in the log:

2016-04-29 21:46:26,760 INFO  AwsSqsReader:215 - .doTheRead()... MESSAGE BREADCRUMB... TYPE=XXXXX    MessageId:     mmmmm    ReceiptHandle: ddddddd    MD5OfBody:     3893393d93835e123d243903b84cfb34    Body:          {
  "Type" : "Notification",
  "MessageId" : "dddddd",
  "TopicArn" : "arn:aws:sns:us-east-1:dasfdasfdsafdasf:asdfasdfasdf",
  "Subject" : "ddddddddddd",
  "Message" : "{ \"EventCode\": 12345,         \"Summary\": \"llllll\",         \"Node\": \"uuuuuuu\",         \"Severity\": 3,         \"EMSInstance\": \"ddddddd\",         \"Agent\": \"aaaaaaa\",         \"AlertGroup\": \"ddd\",         \"AlertKey\": \"ddddd\",         \"Location\": \"ssssssssss\",         \"EpochTime\": \"1461980781\",         \"GenericString1\": \"abbbb\",         \"GenericString2\": \"12345\"         } ",
  "Timestamp" : "2016-04-30T01:46:22.278Z",
  "SignatureVersion" : "1",
  "Signature" : "xxxxxxxx",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-ddddddd",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=xxxxxxxxx"
}

There are two events shown on the search results page, with the split at the '"Timestamp":...' line.

On the splunk-forwarder host, I have tried to create my own sourcetype, reconfiguring the splunk-forwarder to use 'mylog4j' for this file... and then in $SPLUNKHOME/system/local/props.conf:

[mylog4j]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2},\d{3})
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25

I restarted the splunk-forwarder, sent a new event thru to the log, tried to re-search. Alas, I still see two events on search results page. I think the sourcetype change worked, since I had to change 'sourcetype=mylog4j' on the search.

How can I remedy this?

thx in advance
marshall

0 Karma

Jeremiah
Motivator

If you are running a universal forwarder, you need to put this props.conf setting on the indexer.

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...