Getting Data In

Need help on LINE_BREAKER,TIME_FORMAT and TIME_PREFIX

sathiyasun
Explorer

I have built a props.conf but when I upload the log file manually it works fine but when the app writes the log the line break is not working. Please advise how to make this props.conf working when the app writes log file.

Complete Log file:-

INFO:SESSION TOKEN = A5BD11D7B2BB200B5FB893C120333044:2018-08-28 15:12:03,991
INFO:REQUEST:2018-08-28 15:12:08,456
INFO: version="1.0" encoding="UTF-8"?>



      <web:CreateRecord>
         <web:sessionToken>A5BD11D7B2BB200B5FB8C120333044</web:sessionToken>
         <web:moduleId>433</web:moduleId>
         <web:fieldValues><Record><Field id=&quot;17471&quot; value=&quot;sri_id&quot;/><Field id=&quot;16109&quot; value=&quot;result.count=1&amp;#x3C;br /&amp;#x3E;result.index=amt&amp;#x3C;br /&amp;#x3E;result.sourcetype=AMT-n-AGA-SystemErr&quot;/><Field id=&quot;16108&quot; value=&quot;##This is test Alert##  - Testing the Line breaking rule&quot;/><Field id=&quot;17343&quot; value=&quot;15:12:03&quot;/><Field id=&quot;25310&quot; value=&quot;85040&quot;/><Field id=&quot;17339&quot; value=&quot;15:12:03&quot;/></Record></web:fieldValues>
      </web:CreateRecord>

Line Break from this line(INFO:SESSION TOKEN = A5BD11D7B2BB200B5FB893C120333044:2018-08-28 15:12:03,991)

Props.conf

[wsa]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(\w+\:\w+\s\w+\s\=\s\w+\:\d+\-\d+\-\d+\s\d+\:\d+\:\d+\,\d+)
NO_BINARY_CHECK = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = (?i)info\:session[^=]+\=[^:]+\:
pulldown_type = true
disabled = false
MAX_TIMESTAMP_LOOKAHEAD = 30
0 Karma

somesoni2
Revered Legend

Give this a try

[wsa]
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)(?=INFO:SESSION TOKEN \=)
 TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
 TIME_PREFIX = INFO:SESSION TOKEN \=[^\:]+\:
 MAX_TIMESTAMP_LOOKAHEAD = 23

Points 2,3 and 4 from @MuS's comments apply here too.

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi sathiyasun,

just are few hints here:
- the regex matches the line breaker, so that should not be the problem
- did you place the props.conf on the parsing instance (the first full Splunk instance that receives the events)?
- did you restart Splunk after you applied the props.conf?
- Is the sourcetype name correct? The stanza match is case sensitive 😉

Hope this helps ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Building Momentum: Splunk Developer Program at .conf25

At Splunk, developers are at the heart of innovation. That’s why this year at .conf25, we officially launched ...