I have several text format log files in which I need help in linebreaking them into the appropriate events that I need. This is an exmaple of what my file looks like:
====================================================
LOGS_BEGIN: DO_LOG
====================================================
====================================================
-rw------- 1 root root 3505 Apr 23 12:58 /var/adm/sulog
====================================================
SU 02/12 16:53 - pts/1 hkim-root
SU 02/12 16:53 + pts/1 hkim-root
SU 02/13 13:10 + ??? root-sys
SU 02/14 13:10 + ??? root-sys
SU 02/15 10:28 + pts/1 hkim-root
SU 02/15 13:10 + ??? root-sys
SU 02/16 13:10 + ??? root-sys
SU 02/17 13:10 + ??? root-sys
SU 02/18 13:10 + ??? root-sys
SU 02/19 13:10 + ??? root-sys
SU 02/19 16:42 + pts/1 hkim-root
SU 02/20 13:10 + ??? root-sys
SU 02/20 15:06 + pts/1 hkim-root
SU 02/21 09:54 + pts/1 hkim-root
SU 02/21 13:10 + ??? root-sys
SU 02/21 17:22 - pts/1 hkim-root
SU 02/21 17:22 + pts/1 hkim-root
SU 02/22 09:56 + pts/1 hkim-root
SU 02/22 10:03 + pts/1 hkim-hkim
SU 02/22 10:03 + pts/1 hkim-sybmp
SU 02/22 13:10 + ??? root-sys
====================================================
====================================================
I need to be able to filter out the top part of the log,which consist of the 1st line till the 6th line (shown below)
====================================================
LOGSBEGIN: DOLOG
====================================================
====================================================
-rw------- 1 root root 3505 Apr 23 12:58 /var/adm/sulog
====================================================
After which,I will need each and every line to be a seperate event (shown below)
SU 02/15 10:28 + pts/1 hkim-root
SU 02/15 13:10 + ??? root-sys
I would also have to filter out the last part of the log(shown below)
====================================================
====================================================
I have tried several props.conf be it doesn't to be working!! Please help me!
Try this
props.conf
[yoursourcetypename]
SHOULD_LINEMERGE=false
MAX_TIMESTAMP_LOOKAHEAD=15
TRANSFORMS-d1=delete_header_and_footer
transforms.conf
[delete_header_and_footer]
REGEX=(?:^=====|^LOGSBEGIN|^[-rw][-rw][-rw])
DEST_KEY=queue
FORMAT=nullQueue
What it does: tells Splunk that your data is one event per line, with the timestamp in the first 15 characters. It also takes each event and checks it against the regular expression in transforms.conf. Events that match are discarded.
I checked the regular expression at http://gskinner.com/RegExr/ but you should check it too.
And BTW, the "d1" is just an arbitrary identifier, but it needs to be unique within props.conf.
You could simply remove this line
MAX_TIMESTAMP_LOOKAHEAD=15
I don't think it is really necessary
Hi,thanks for the help!
I hav another questn,my file also includes these lines:
====================================================
====================================================
-rw------- 1 root sys 2984 Apr 11 2006 /var/adm/loginlog
====================================================
vkaliya:/dev/pts/1:Wed May 4 11:14:34 2005
vkaliaya:/dev/pts/1:Wed May 4 11:14:43 2005
Your above prop.conf and transform.conf works with the earlier example,but it would not correctly timestamp the events (in the 2nd example). Can you assist me pls?