Getting Data In
Highlighted

how to split log file on non-standard date and special characters

Path Finder

Hi All,

I have a log file that has a non standard date/time and special characters and i am trying to split the lines on the forwarder, however i cannot get Splunk to read anything less than 257 lines at a time!

My Log lines are as follows;

[Wed May 31 12:34:22.431862 2017] [:debug] [pid 10890] ils_util.c(615): SSS_DBG:[10890,55584809]:-  
+------------------- Start Of Request --------------------
| Id = 55584809,  SessionId = 1300,  Sid = 28716112
+---------------------------------------------------------
[Wed May 31 12:34:22.431867 2017] [:debug] [pid 10890] ils_util.c(618): SSS_DBG:[10890,55584809]:-  Start Of Request - Id = 55584809,  SessionId = 1300,  Sid = 28716112

my props.conf file is below

[interrorlog]
SHOULDLINEMERGE = true
BREAK
ONLYBEFOREDATE = false
BREAKONLYBEFORE = ^[\d{3}\s\d{3}\s\d{2}\s\d{2}:\d{2}:\d{2}.\d{6}\s\d{4}]\s
MAXTIMESTAMPLOOKAHEAD = 35
TIME_PREFIX = ^

i have tried to use LINE_BREAKER=([+||]+) in order to deal with the special characters, however this resulted in even more lines per break.

thanks

Tags (2)
0 Karma
Highlighted

Re: how to split log file on non-standard date and special characters

SplunkTrust
SplunkTrust

First you need to setup the event parsing (line breaking, timestamp identification etc) on your indexers/heavy forwarders. If the instance monitoring this is Universal forwarders, then move the line breaking configurations to your Indexers/heavy forwarders (a restart of Splunk would e needed).

Second, give this config a try

[int_error_log]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\[\w+\s+\w+\s+\d+\s+\d+\:\d+\:\d+\.\d+)
TIME_PREFIX = ^\[\w+\s+
TIME_FORMAT = %b %d %H:%M:%S.%N %Y
MAX_TIMESTAMP_LOOKAHEAD = 27

View solution in original post

0 Karma
Highlighted

Re: how to split log file on non-standard date and special characters

Path Finder

Thanks somesoni2,
Worked perfectly 🙂 - apologies for delay in my reply have been away 🙂

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.