Getting Data In

Need help on LINE_BREAKER,TIME_FORMAT and TIME_PREFIX

sathiyasun
Explorer

I have built a props.conf but when I upload the log file manually it works fine but when the app writes the log the line break is not working. Please advise how to make this props.conf working when the app writes log file.

Complete Log file:-

INFO:SESSION TOKEN = A5BD11D7B2BB200B5FB893C120333044:2018-08-28 15:12:03,991
INFO:REQUEST:2018-08-28 15:12:08,456
INFO: version="1.0" encoding="UTF-8"?>



      <web:CreateRecord>
         <web:sessionToken>A5BD11D7B2BB200B5FB8C120333044</web:sessionToken>
         <web:moduleId>433</web:moduleId>
         <web:fieldValues><Record><Field id=&quot;17471&quot; value=&quot;sri_id&quot;/><Field id=&quot;16109&quot; value=&quot;result.count=1&amp;#x3C;br /&amp;#x3E;result.index=amt&amp;#x3C;br /&amp;#x3E;result.sourcetype=AMT-n-AGA-SystemErr&quot;/><Field id=&quot;16108&quot; value=&quot;##This is test Alert##  - Testing the Line breaking rule&quot;/><Field id=&quot;17343&quot; value=&quot;15:12:03&quot;/><Field id=&quot;25310&quot; value=&quot;85040&quot;/><Field id=&quot;17339&quot; value=&quot;15:12:03&quot;/></Record></web:fieldValues>
      </web:CreateRecord>

Line Break from this line(INFO:SESSION TOKEN = A5BD11D7B2BB200B5FB893C120333044:2018-08-28 15:12:03,991)

Props.conf

[wsa]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(\w+\:\w+\s\w+\s\=\s\w+\:\d+\-\d+\-\d+\s\d+\:\d+\:\d+\,\d+)
NO_BINARY_CHECK = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = (?i)info\:session[^=]+\=[^:]+\:
pulldown_type = true
disabled = false
MAX_TIMESTAMP_LOOKAHEAD = 30
0 Karma

somesoni2
Revered Legend

Give this a try

[wsa]
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)(?=INFO:SESSION TOKEN \=)
 TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
 TIME_PREFIX = INFO:SESSION TOKEN \=[^\:]+\:
 MAX_TIMESTAMP_LOOKAHEAD = 23

Points 2,3 and 4 from @MuS's comments apply here too.

0 Karma

MuS
Legend

Hi sathiyasun,

just are few hints here:
- the regex matches the line breaker, so that should not be the problem
- did you place the props.conf on the parsing instance (the first full Splunk instance that receives the events)?
- did you restart Splunk after you applied the props.conf?
- Is the sourcetype name correct? The stanza match is case sensitive 😉

Hope this helps ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...